Customize Your Environment with Chef Recipes

Updated:

You can use Chef recipes to customize your Engine Yard Cloud environments. You can edit cookbooks that are provided by Engine Yard or you can make your own cookbooks.

This page describes:

Set up the Chef environment

In order to be able to develop Chef recipes that build (and rebuild each time you start an instance) your environment customizations, you need to install the engineyard gem in your local environment on your development machine.

Even if you already have the engineyard gem installed, it is good practice to run "install engineyard" to make sure that you have the latest version of the gem.

To install the engineyard gem

  1. Type:

    $ sudo gem install engineyard 
  2. When prompted, enter the password for your Engine Yard account.

Clone the ey-cloud-recipes repository

To work with the ey-cloud-recipes repository, you need to fork and clone it to your development environment on your local machine. Clone a local copy of the ey-cloud-recipes repository in the directory that you'll work in when writing custom Chef recipes for your environment (the same environment where you installed the engineyard gem).

Here is the standard procedure for cloning a GitHub repository:

To fork and clone the ey-cloud-recipes repository

  1. Browse to the ey-cloud-recipes site on GitHub.
  2. Click Fork to fork the repository.
    This creates a fork under your user account on GitHub.
  3. Copy the URL of your forked repository to your clipboard.
  4. Clone the repository to your local development machine:
    Note: Unless you are using submodules in git, do not put the ey-cloud-recipes repository in the same directory as your application repository. By default, git does not support nesting.

About updating the ey-cloud-recipes repository

From time to time, you might want to or need to refresh your ey-cloud-recipes repository to keep up with changes made by Engine Yard. A good reason to refresh the repository is if something isn't working correctly or if there is a new feature that you want to take advantage of. You can review the GitHub Commit History for the repository to find out about recent changes.

Important! Be sure to test carefully after updating the repository and before applying new Chef recipes to a production environment.

About the file structure of ey-cloud-recipes

Become familiar with the file structure of the ey-cloud-recipes repository.

Base files and directories

README.md 
Rakefile
cookbooks/
main/
attributes/
definitions/
libraries/
recipes/

Cookbooks directory

In the cookbooks directory are directories of the self-contained recipes for various "components" (such as Sphinx, MongoDB, Redis, Varnish) that you can enable and customize for your environment.

Each directory under a cookbook has a number of sub-directories. On this page, we use the Sphinx recipe as the example:

cookbooks/ 
sphinx/
files/
default/
sphinx.logrotate
recipes/
default.rb
templates/
default/
sphinx.monitrc.erb
sphinx.yml.erb

Recipes directory

For each cookbook `recipes/default.rb is the main definition file that prescribes how Chef performs each of its actions to achieve the customization. View the Sphinx example.

Files directory

In the Sphinx cookbook, the recipes/default.rb creates a remote_file resource (that's a Chef term). That resource is found in the files/default/sphinx.logrotate location and corresponds to the source value in the code block below.

remote_file "/etc/logrotate.d/sphinx" do 
owner "root"
group "root"
mode 0755
source "sphinx.logrotate"
backup false
action :create
end

The sphinx.logrotate is a file that has no variables. You use the templates and ERB to insert the variable data into the sphinx.logrotate file.

Templates directory

Also in the the Sphinx cookbook, the recipes/default.rb creates a template and passes variables to the template.

In the recipes/default.rb file:

template "/etc/monit.d/sphinx.#{app_name}.monitrc" do 
source "sphinx.monitrc.erb"
owner node[:owner_name]
group node[:owner_name]
mode 0644
variables({
:app_name => app_name,
:user => node[:owner_name],
:flavor => flavor
})
end

The variables above are passed to the template (sphinx.monitrc.erb), and Chef renders the static file (/etc/monit.d/sphinx.myapp.monitrc).

In the sphinx.monitrc.erb file:

check process sphinx_<%= @app_name %>_3312 
with pidfile /var/run/sphinx/<%= @app_name %>.pid
start program = "/engineyard/bin/<%= @flavor %>_searchd <%= @app_name %> start" as uid <%= @user %> and gid <%= @user %>
stop program = "/engineyard/bin/<%= @flavor %>_searchd <%= @app_name %> stop" as uid <%= @user %> and gid <%= @user %>
group sphinx_<%= @app_name %>

Turn on a cookbook

In order to turn on a cookbook and have that set of recipes run when you deploy your application, you need to uncomment the recipe in the cookbooks/main/recipes/default.rb file.

To turn on an existing cookbook

  1. Select the cookbook that you want to use.

  2. In your local environment, open cookbooks/main/recipes/default.rb for editing.

  3. Uncomment the "include_recipe" line for the recipe that you want to turn on.

  4. Follow any additional instructions in the comments.
    For example, for Sphinx, you need to edit cookbooks/sphinx/recipes/default.rb; for details, see Implement full text search with Sphinx on Engine Yard Cloud.

  5. Save cookbooks/main/recipes/default.rb and commit your changes locally and push them to your remote repository.

  6. Use the ey recipes commands to upload and apply the recipes:

    ey recipes upload -e environment_name 
    ey recipes apply -e environment_name

Create a cookbook

If there isn't a ready-made cookbook to suit your needs, you can create your own cookbook from scratch.

The procedure below shows how to create a new Chef recipe called nginx_logrotate. This recipe changes the retention time of Nginx logs from 30 days (the default) to 60 days.

To generate a new Chef recipe:

  1. In your local environment, in the ey-cloud-recipes repository directory, type:

    $ rake new_cookbook COOKBOOK=nginx_logrotate 

    This creates the file: cookbooks/nginx_logrotate/recipes/default.rb.

  2. Edit cookbooks/nginx_logrotate/recipes/default.rb to add the following code:

    remote_file "/etc/logrotate.d/nginx" do 
    owner "root"
    group "root"
    mode 0755
    source "nginx.logrotate"
    backup false
    action :create
    end
  3. Create a file called files/default/nginx.logrotate with the following content:

    /var/log/engineyard/nginx/*.log { 
    daily
    missingok
    compress
    rotate 60
    dateext
    notifempty
    sharedscripts
    extension gz
    postrotate
    [ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`
    endscript
    }
  4. Edit cookbook/main/recipes/default.rb to enable the recipe:

     include_recipe "nginx_logrotate" 
  5. Test the syntax of your new recipe:

     $ rake test 
  6. Commit your changes locally and push them to your remote repository.

  7. Use the ey recipes commands to upload and apply the recipes:

        ey recipes upload -e environment_name 
    ey recipes apply -e environment_name

Report to the dashboard from custom recipes

You can have messages appear when your custom Chef recipes run. These appear on the Environment page under the "Instances" heading; and allow you to see when the custom portions of your Chef recipes are running. These messages also appear in the chef.custom.log file. (See Report to a log file from custom Chef recipes below.)

To report to the dashboard from custom Chef recipes

  1. Edit the Ruby file (for example, recipes/default.rb) file with code like this:

    ey_cloud_report "recipe_name" do 
    message "message text"
    end

    Where more message text is the message that you want to appear on dashboard when that part of the code is executed.

    For example:

    ey_cloud_report "nginx" do 
    message "custom logrotate for nginx"
    end

There is an example of the ey_cloud_report method in the Sphinx recipe.

Report to a log file from custom Chef recipes

Custom Chef recipes are logged to /var/log/chef.custom.log. (Default Engine Yard Cloud recipes are logged to /var/log/chef.main.log.)

To report to a log file from custom Chef recipes

  1. Edit the RB file (for example, recipes/default.rb) file with code like this:

    Chef::Log.info "message text" 
    For example:
    Chef::Log.info "Doing step 1." 
    writes a line like this:
    [Sun, 22 Jan 2012 22:29:00 +0000] INFO: Doing step 1. 

    to the /var/log/chef.custom.log file.

Specify which instance roles run a recipe

In a clustered environment, you have multiple instances, each instance playing a different role. In most cases, you want the recipe to run on only one type of instance; for example, to run on the application master, but not on the application slave, database, or utility instances.

To specify which instance (role) runs a recipe

  1. Add an if statement around the recipe code:
    if node[:instance_role] == 'instance_role' 
    Where instance_role is one of the following:
    • app_master (for the application master)
    • app (for an application slave)
    • solo (for a single instance)
    • db_master (for a database master)
    • db_slave (for a database slave)
    • util (for a utility instance)

Examples

In the ssh_tunnel recipe, to have the recipe run on only the application master in a clustered environment:

  if node[:instance_role] == 'app_master' 
...
end

However, to have the same recipe run in either a clustered environment or a single-instance environment, write the condition this way:

  if ['app_master', 'solo'].include?(node[:instance_role])
...
end

About utility servers

If you want a recipe to run on a particular utility server, you can specify it by name instead of by instance role.

For example,

  if node[:name] == 'myutility' 
...
end

Where myutility is the name of a utility instance.

More information

For more information about...See...
the engineyard gem                                                                          Engine Yard CLI User Guide.
the ey recipes commands Engine Yard CLI User Guide.
custom Chef recipes Custom Chef Recipes examples & best practices by Tim Littlemore.
running custom Chef recipes during deploy Custom Chef Recipes During Deploy blog by Dr. Nic Williams.

If you have feedback or questions about this page, add a comment below. If you need help, submit a ticket with Engine Yard Support.

Was this article helpful?
22 out of 22 found this helpful
Have more questions? Submit a request

Comments

  • Avatar
    Jesse Cooke

    I had a very simple recipe that was missing main/attributes/recipe.rb, and my chef recipes would not run. That file is needed by chef to run, and should include the following 3 lines:

     

    recipes 'main'

    owner_name @attributes[:users].first[:username]

    owner_pass @attributes[:users].first[:password]

  • Avatar
    Shai Rosenfeld

    It should be in ey-cloud-recipes, which is the repo for our example recipes: recipe.rb

  • Avatar
    Keri Meredith

    Hi Jesse, thanks for your input. The missing recipe.rb file is part of the Clone the ey-cloud-recipes repository section above.

    As Shai mentioned, you can find recipe.rb in ey-cloud-recipes on Github.

    Best practice is to clone it and add custom recipes after that. kjm

  • Avatar
    Petteri Räty

    I wonder why the instructions tell you to push your changes to github when that is not required for ey recipes to work. It's of course a form of backup and contribution to the general community but people should be told it's their choice.

  • Avatar
    Marcus Malmberg

    May I suggest that you add, to the guide, that one need to have the recipe.rb file containing:

    recipes 'main'

    owner_name @attributes[:users].first[:username]

    owner_pass @attributes[:users].first[:password]

    in order to be able to run any recipe at all. I read Jesse's comment earlier but didn't realize that it was needed to run any recipes at all, and thought it was related to something with his code. 

    I'm aware that this guide want you to use ey-cloud-recipes, but I was only interested in a simple recipe that would create a swapfile and it felt superfluous to clone a git repo to accomplish that. Since it's so easy to create the needed folder structure and files manually I'd love to see a short section about that in the guide as well, since I imagine that I'm not alone in this matter :)

  • Avatar
    Jason Fowler

    There is a typo under Cookbooks Directory:  "teplates" should be "templates".  Mmmm..

    teplates/

  • Avatar
    Diana Lam

    Thanks for catching that Jason! Typo has been fixed.

  • Avatar
    Sébastien Grosjean

    Thanks to @dvalfre on IRC, I could find how to get the current node[:ec2][:instance\_type] from the console. Simply login into your instance and do `curl "

    http://169.254.169.254/latest/meta-data/instance-type"`

  • Avatar
    Seth Berger

    I think you mean uncomment "include_recipe" instead of "require_recipe"

  • Avatar
    Diana Lam

    Thanks Seth. I've verified this with our cookbook guru and made the change.

  • Avatar
    J. Austin Hughey

    It occurs to me that folks may want to find environment info for their custom chef code. I wrote an article on this that should help you out if that's your current objective:

    https://blog.engineyard.com/2014/engine-yards-chef-node-object

    This explains how to find stuff like the instance name, the instance role, the "framework environment" (e.g. production, test, development, staging, etc.) and so on. Also includes a full documenting of the dna.json file, all keys and values and what they mean, as found on the instance.

  • Avatar
    Tom Giles

    I have a windows development server with MinGW installed.  However running

    rake new_cookbook COOKBOOK=nginx_logrotate

    gives a syntax error because /p is not a valid flag for DOS's mkdir command, and it doesn't realise it should use MinGW for this.  How might I get round this? I've tried manually creating the first folder which causes the error, but it still errors...

     

  • Avatar
    J. Austin Hughey

    Tom, I'm just now seeing your comment even though it says it was a while ago when you wrote it, so hopefully you already got a fix somehow, but the gist of this is that $PATH for your shell (probably bash if you're using mingw, but if not using bash/zsh, that's definitely problem number one, never use Windows default "shell" (and I use that term loosely!) when using mingw/cygwin/etc.) is finding the Windows version of mkdir before the *nix version. You need to modify $PATH to find the mingw version of mkdir before it finds the Windows version.

Please sign in to leave a comment.