Migrating Custom Chef Recipes from v4 to v5

NOTE: When migrating to v5 we recommend that you boot a new environment and test your application there first. We do not recommend an in-place upgrade from v1/v2/v4 to v5.

Chef on v5 differs from earlier stacks, in that there are no longer two distinct main and custom runs, just a single combined run. This combined run is however still initiated via the Dashboard Apply button or the ey recipes apply command. For full details on Chef on v5 please see this article.

Custom chef recipes themselves function differently on v5. All custom chef recipes that work on v2 and v4 need to be updated.

Downloading Existing Recipes

If you do not already have an up to date copy of the recipes in use on your v4 environment, then these can be downloaded via the CLI, using the the ey-core gem:

If not already installed:

gem install ey-core

then:

ey-core recipes download --environment <nameofenvironment>

EY Authored Recipes

If you previously made use of Engine Yard authored custom Chef recipes on v4, then please firstly check the v5 custom cookbooks repository for the equivalent recipe. Most recipes will assume you have the base structure required, that being the ey-custom cookbook, which is invoked as a part of the combined Chef run.

To do this complete the following: 

  1. Create the cookbooks/ directory. You can create this directory anywhere on your local system, preferably within a directory you will associated with the specific environment(s) the recipes will be run on. Some customers may then commit this directory to a repository for easy management. Other customers choose to add the cookbooks/directory to the root of your application, though this should be noted that doing so does not automate the running of the cookbooks in any way, it is simply for ease of storage. Optional - If you have an existing cookbooks/ directory in your working directory, move it to a temporary directory first.

      cd /path/to/working_dir
      (mv -i cookbooks cookbooks.v4)
      mkdir cookbooks
    
  2. Create the required cookbook files:

    mkdir -p cookbooks/ey-custom/recipes
    touch cookbooks/ey-custom/recipe/after-main.rb
    touch cookbooks/ey-custom/metadata.rb

    ey-custom/recipes/after-main.rb is now the entry-point of custom chef recipes. You do not need the main/ directory that you used on v4. Edit the recipe's metadata file as below:

      # cookbooks/ey-custom/metadata.rb
      name 'ey-custom'
  3. For each custom recipe you wish to utilise, follow that specific recipe's readme, but fundamentally you will need to include the recipe in ey-custom's after-main.rb and depend on it in the metadata.rb. Most recipes contain a copy of ey-custom in their cookbooks directory, which can be copied over to avoid steps 1-2 if this is the first custom recipe being utilised, or the contents of which should be merged with the existing ey-custom recipe if this is an additional recipe.

    For example, to use the delayed_job4 recipe that EY maintains, you could copy both ey-custom and custom-delayed_job4 from https://github.com/engineyard/ey-cookbooks-stable-v5/tree/next-release/custom-cookbooks/delayed_job4/cookbooks to your cookbooks/ directory if you did not already have an ey-custom cookbook, or if you did, just copy the custom-delayed_job4 directory and then:

    On cookbooks/ey-custom/recipes/after-main.rb, add

      include_recipe 'custom-delayed_job4'

    On cookbooks/ey-custom/metadata.rb, add

      depends 'custom-delayed_job4'

If you have no self-authored recipes then skip ahead to Applying Recipes.

Customer Authored Recipes

If you want to convert your existing v4 custom chef recipes to v5 because we have not converted them yet or they are recipes you wrote yourself, follow steps 1 & 2 of the above EY Authored Recipes section in order to backup the existing recipes and create the base structure, then complete the following steps:

  1. Copy the recipe from cookbooks.v4/ to cookbooks/. If you follow our instructions above, you will start with an empty cookbooks/.

    cd /path/to/working_dir
     cp -pr cookbooks.v4/RECIPE_NAME cookbooks/
    
  2. Add cookbooks/RECIPE_NAME/metadata.rb.

      name 'RECIPE_NAME'
  3. Add include_recipe and depends to the ey-custom recipe. You always need to add these 2 for any recipe that you want to use.

    On cookbooks/ey-custom/recipes/after-main.rb, add

      include_recipe 'RECIPE_NAME'

    On cookbooks/ey-custom/metadata.rb, add

      depends 'RECIPE_NAME'
  4. Replace node['instance_role'], node['name'], and similar code and add ['dna']. We moved some data on dna.json from the "root" to dna{}. Check out/etc/chef/dna.json on a v5 instance to see the new format.

    Run the code below to view the code that needs to be changed

      cd /path/to/cookbooks.v4/ # or cookbooks/
      grep -r "node\[\(:\|'\|\"\)\(applications\|engineyard\|environment\|instance_role\|members\|name\|users\|utility_instances\)" .
    
      # node['instance_role'] on v4
      node['dna']['instance_role']
    
      # node ['name'] on v4
      node['dna']['name']

    Data specified on the attributes directory will not change. If you have

      default['redis'] = {'version' => "3.2.1"}

    you would still use

      node['redis']['version']
  5. Change the package versions. The v5 stack has new packages. Update your recipes to use these new versions. Refer to the table of common packages below.

    Package Package name V5 Version
    Imagemagick media-gfx/imagemagick 6.9.4.1
    Java dev-java/icedtea-bin 3.3.0 (JDK 8)
    7.2.6.8 (JDK 7)
    Memcached net-misc/memcached 1.4.25
    Redis dev-db/redis 3.2.0
    wkhtmltopdf media-gfx/wkhtmltopdf-bin 0.12.2.1-r1
  6. (Optional) Use a string instead of symbol when accessing node attributes. The code below are similar. Either can be used. Foodcritic (lint tool for Chef) suggests using a string.

      # use this
      node['dna']['instance_role']
    
      # don't use this
      node[:dna][:instance_role]

Applying Recipes

Once the recipes are ready upload them to environment via the CLI, using the ey-core gem:

If not already installed:

gem install ey-core

then:

ey-core recipes upload --environment <nameofenvironment>

then run Chef via the dashboard Apply button or via the CLI:

ey-core recipes apply --environment <nameofenvironment>

Comments

  • Avatar
    Dan Moore
    0
    Comment actions Permalink
  • Avatar
    Rad Batnag

    Thanks Dan! I updated those links.

    0
    Comment actions Permalink
  • Avatar
    Gabe da Silveira

    Step 3 is incongruous: the second paragraph says you don't have to copy cookbooks, and then the third paragraphs says to do just that....

    0
    Comment actions Permalink
  • Avatar
    Ralph Bankston

    Hi Gabe. Thanks for pointing that out. Updated the documentation a bit to hopefully be more clear that some recipes don't need to have their cookbooks copied

    0
    Comment actions Permalink
  • Avatar
    Gabe da Silveira

    @Ralph, it still says:

    "For example, to use the delayed_job4 recipe that EY maintains, you only need to copy custom-delayed_job4 on https://github.com/engineyard/ey-cookbooks-stable-v5/tree/next-release/custom-cookbooks/delayed_job4/cookbooks to your cookbooks/ "

    0
    Comment actions Permalink
  • Avatar
    Ralph Bankston

    Hi @Gabe updated the text to read:

    `You can use some custom chef recipes without copying the actual recipes. The list of Engine Yard supported recipes can be found here.`

    0
    Comment actions Permalink
  • Avatar
    Gabe da Silveira

    @Ralph, sorry if I'm being dense, but under step 3 it still says:

    "For example, to use the delayed_job4 recipe that EY maintains, you only need to copy custom-delayed_job4 on https://github.com/engineyard/ey-cookbooks-stable-v5/tree/next-release/custom-cookbooks/delayed_job4/cookbooks to your cookbooks/ directory."

    But my understanding is that you DON'T need to do that. Am I misunderstanding something?

    Edited by Gabe da Silveira
    0
    Comment actions Permalink
  • Avatar
    Tien Nguyen Quang

    sorry, no bad feeling, i'm surprise nobody brought it but this is by far, the worst documentation I've been through

    I would be very appreciated if we can replace the whole documentation with a link to an actual documentation where I can actually do something and understand it

    Edited by Tien Nguyen Quang
    3
    Comment actions Permalink
  • Avatar
    Claudio Mendoza

    I agree with Tien that this should be re-written preferably by someone that really knows what is going on.

    0
    Comment actions Permalink
  • Avatar
    Dustin Schaerer

    I am also having trouble understanding these instructions. Can we please have these instructions rewritten?

    0
    Comment actions Permalink
  • Avatar
    Paul Tingle

    Thanks everybody for your feedback. Apologies for the delay, but this article has now been expanded upon and hopefully clarified, so please let us know any further feedback on it.

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk