Note: since this article was originally written, the world of software development turned into using environment variables for almost everything. So the feature was added to Engine Yard. Nevertheless, the contentes of this article remain relevant.
Using environment variables is a somewhat common practice during Development but it is actually not a healthy practice to use with Production. While there are several reasons for this, one of the main reasons is that using environment variables can cause unexpected persistence of variable values. For example:
When using environment variables with Unicorn, hot restarts don't pick up changes to environment variables. This is because the previous master process starts the new master process, which retains the same environment variables. To pick up new environment variable values, you must fully terminate the old Unicorn master process and start a new one using a wrapper script (which sets the environment variables to the new values). Unfortunately, this causes downtime, so deploys only use the hot restart method.
While choosing between no downtime and stale environment variables is problematic, the bigger issue is using environment variables in the first place.
Note: Ruby programmers will be familiar with the using the "ENV" hash to access environment variables; other languages may use different methods, however the concept is the same.
Environment variables not recommended for every situation
Environment variables provide a good way to set application execution parameters that are used by processes that you do not have direct control over. However, environment variables should not be used for configuration values within your own dynamic applications.
Environment variables are global variables. A best practice for most programming languages and design paradigms is to avoid the use of global variables. Furthermore, there is a security risk in setting tokens in environment variables. All gems used and sub-processes launched have access to all variable values, so if any of them log or transmit the output of 'export' or ‘env’, your private data can be exposed.
Another challenge to environment variables is scrubbed environments. For security reasons, cron and monit don't start processes with the environment variables provided by the user's login profile. If you're relying on environment variables, it can lead to frustration and confusion when the process works when you start in manually, but not when cron or monit does.
Instead of environment variables, we recommend that you either use a YAML configuration file or a shared data source.
YAML files as an alternative to environment variables
Instead of environment variables, it’s best practice to handle configuration parameters in a YAML file that is stored in /data/<app>/shared/config and linked to /data/<app>/current/config during the deploy process, using the before_migrate.rb deploy hook. For more information, see Use Deploy Hooks.
Tip: Try using the settingslogic gem to provide the YAML settings logic. It neatly wraps up all the functions needed. Restarting the app will pick up the changes and can be done gracefully (ie using a hot restart with no downtime).
Shared data source as an alternative to environment variables
If you find yourself needing to change these values often, we recommend a shared data source, such as a parameters table on your existing DB; if you need something faster, we recommend a Redis or Riak source. A custom recipe or admin tool can be written to populate these values and changed as needed without deploying or restarting the app.
Ensure data availability
One of the nice things about environment variables is that they are always available, right from the start. However, you can have the same benefits when using loaded data. You just need to read it early using initializers. This article shows several ways to accomplish this. We often recommend using config/environment.rb to load the config before Rails invokes the other initializers. However, the recommended method may vary depending on your situation.
If you have feedback or questions about this page, add a comment below. If you need help, submit a ticket with Engine Yard Support.
In instances where you are standing up multiple environments and do not want to be limited by the "dev, staging, production" standards, environment variables allow you to generate flexible and infinite variations of your environment template.
Particularly, in light of the unicorn hot-start situation you described, I'm curious what, you recommend for configuring something like Ruby 2.x's GC settings?
As I understand it those settings are controllable via ENV vars. (Not sure if there are other ways to tweak them or if it is even possible to effect them (e.g. RUBY_HEAP_MIN_SLOT, RUBY_GC..., etc) by setting them _after_ a ruby program is running (e.g. ENV['RUBY_HEAP_MIN_SLOT'] = 11). Nevertheless, from your article it sounds like that is risky, given the potential for other code or situations to cause those settings to get cleared.
thoughts?
This advice to use the production database for storing configuration settings just doesn't work, because in a Rails app you can't access the database (yet) when the app is starting, when you need to be able to read your configuration settings.
You can't read the database when config/environments/production.rb is processed, so how are you supposed to set up something like your SMTP username/password for ActionMailer::Base.smtp_settings ?
How are you supposed to set up the secret_key_base in your config/secrets.yml? You can't use the database because you have a chicken-and-egg problem. You can't even migrate your database to create your settings table during a production deployment because your config/secrets.yml file will throw an error because the settings table doesn't exist -- when you're trying to create it.
Engine Yard needs some solution that provides environment variables. The Rails world uses environment variables for this, and trying to do it with the database is not straightforward at all. There needs to be a "configuration" section in the environment settings or something. With Rails 4.1 and beyond, it's increasingly difficult to deploy a very simple Rails app to Engine Yard.
If you're not going to provide a configuration mechanism, then you really need to provide some documentation on how you think that people should handle things like configuring the secret_key_base. "Put it in the database" doesn't work. Please try it and see for yourself. Just try creating a plain Rail 4.2 app and try to deploy it to Engine Yard. You won't even be able to deploy it without dealing with this secret_key_base problem in some way, and your suggested solution doesn't work.
Ryan, Asia mentioned above, you can use YAML files for configuration and these can be written with custom Chef recipes. Check out the api-keys-yml recipe in http://github.com/engineyard/ey-cloud-recipes. Additionally, if you do want to use Environment Variables, you can check out this blog post https://blog.engineyard.com/2015/use-environment-variables-engine-yard-cloud-envyable
As is, not "Asia"
There's some measure of disagreement with the "not using env vars" approach out there. If you -do- want to use environment variables, I've been working on something in some free time that, while far from finished, might be of interest to you: https://github.com/jaustinhughey/seekrit
The idea is to run this on some instance in your cluster with access to Redis. Then, you run it on a high port number NOT available to the outside world (absolutely not on 80/81/443/444/22 on EY). Be sure that it can't be reached outside of your cluster - absolutely key.
Then you can use the API to set and retrieve secrets from the system. You'll get JSON back.
I have a number of planned improvements in mind; a front-end is in the works to make management easier, and I'd like to experiment with the idea of building a simple script that you could execute like: SUPER_SECRET_KEY=
seekrit\_get super\_secret\_keybundle exec ...Again, I stress that this is untested in reality (it has tests, just hasn't been used in prod yet) and it's under development still, and not an official EY project. Just a tool that you're welcome to use if you want.
The link to settingslogic should be updated to its new repo location: github.com/settingslogic/settingslogic
Evan, thank you for the response. The problem with using Chef to set environment variables is that it would result in our configuration variables being stored in a Git repository on GitHub. All that accomplishes is moving the sensitive information out of one GitHub repo and into another. We might as well just hard-code sensitive settings directly into our Rails apps.
This really sucks but I'm exploring using a lightweight ORM like Sequel along with Active Record for accessing settings in the production DB during the Rails app initialization.
The only time we've been tempted to set environment variables in the application is for locale settings. For some reason most of our QA servers have a LANG variable of "en_US.utf8" while production servers have LANG unset and the rest of the locale settings as POSIX. They're on the same stacks, so they should be the same. But maybe there's some setting I missed.
Our application got to production before we realized it won't allow non-ASCII characters and we don't want to change locale at the server level because who knows what that could break. So ENV[] seems like a good option.
In this situation though, this is something that really should have been set at the server level and never changed.
Mr.Bell is hiding important facts in this article. He states the cons of using environment variables but misses the pros, such as low possibility of adding sensible data to repository or using only one format, which is easy to understand. In order to make a valid decision, one should know the two sides of the medal.