Knowledge Base/Engine Yard Cloud Documentation/Deploy your Application

Use Deploy Hooks

Engine Yard
posted this on February 16, 2012 10:03 AM

Updated: April 1st, 2014

Deploy hooks are Ruby scripts that you write which are executed at designated points in the deployment process. This allows you to customize the deployment of your application to meet its particular needs.

For example, if your application uses Resque, then deploy hooks provide a way for you to restart your Resque workers when you deploy a new version of your application. Airbrake users can use deploy hooks to notify Airbrake of a deploy.

Deploy hooks live in the APP_ROOT/deploy directory of your application. The order in which they run is specified in the documentation for the ey deploy command.

Important: When you add a new instance, Engine Yard automatically runs the deploy hooks. input_ref will not be available and the deploy hooks will fail if you use it. This is a known issue and we are working on a fix. In the meantime, confirm that input_ref is not nil and repo is nil.

Capistrano similarities

Our deployment system mimics Capistrano’s filesystem layout so it will be familiar to you if you have ever used Capistrano. In fact, it is still backwards compatible with Capistrano so you can use both at the same time if so desired.

Structure and sequence

To use deploy hooks, create an APP_ROOT/deploy directory in your application and save named hook files in this directory which will be triggered at the appropriate times during the deployment process. The files are defined as follows and run in the order listed:

APP_ROOT/ 
deploy/
before_bundle.rb
after_bundle.rb
before_compile_assets.rb
after_compile_assets.rb
before_migrate.rb
after_migrate.rb
before_symlink.rb
after_symlink.rb
before_restart.rb
after_restart.rb

Remember that, in order for migrations to run, your entire environment is loaded. So if you have any symlinks that need to be created in order for the application to start properly, put them in before_migrate.rb instead of before_symlink.rb, because before_symlink.rb runs after the migration.

Any deploy hooks that you have defined are called, even if they are hooking into a step that is not necessary for the deployment. For example, after_migrate is called even if there are no new migrations in your deployment.

Shell commands

Besides the usual ruby syntax, you also have access to run, run!, sudo, and sudo! to run commands. See Deploy Hooks API.

CommandRuns as   Exit zero   Non-zero exit    
run deploy user returns true returns false
run! deploy user returns true aborts deploy
sudo root returns true returns false
sudo! root returns true aborts deploy

For example:

run “echo ‘config.release_path: #{config.release_path}’ >> #{config.shared_path}/logs.log”  
run “ln -nfs #{config.shared_path}/config/foo.yml #{config.release_path}/config/foo.yml”
sudo “echo ‘sudo works’ >> /root/sudo.log”

Calling git commands

Here’s an example where you can call a git command from a deploy hook:

run "exec ssh-agent bash -c 'ssh-add /home/deploy/.ssh/<app>-deploy-key && git clone git@github.com:foo/bar.git #{current_path}/tmp/foo'"

Replace <app> with the name of your app and change the path for your git repository.

Deploy hooks API

Deploy hooks are plain old Ruby code. However, the engineyard gem makes some additional methods available within deploy hooks to simplify common tasks.

Note: Engine Yard has deprecated the use of the @configuration variable directly. Instead,use the config object for compatibility with future releases.

The following methods should be accessed through the config object:

  • config.account_name

    The name of the account that owns the environment where the application is deploying: e.g. myaccount

  • config.all_releases

    Array of paths to the deployments on this instance, ordered from oldest to newest.

  • config.app

    This is the name of the application: e.g. myapp

  • config.current_name

    On a utility instance, the name of the utility instance. On non-utility instances, nil.

  • config.current_path

    Path to the current deployment. This is the deployment pointed to by the current symlink, so the value of current_path() will change after the symlink step is executed.

  • config.current_role

    The role of the current instance. Possible values:

    • solo: the sole instance for the environment
    • app_master: the application master
    • app: a non-master application server
    • util: a utility server. The current_name method will allow you to distinguish between different types of utility servers.
  • config.deployed_by

    The name of the user who triggered the deployment: e.g. John Smith.  Not available on rollback.

  • config.framework_env

    The value of the RAILS_ENV, RACK_ENV, and MERB_ENV environment variable.

  • config.environment_name

    This is the name of the environment that the application is deploying on: e.g. myenvironment or myapp_staging.

  • config.input_ref

    The name of the branch that was given as input to the deploy: e.g. master or branch.  Not available on rollback.

    Note:Roll back is ideal for simple deployments with no database migrations or complex deploy hooks. When a data transformation has been deployed, rolling forward by deploying a corrected release may be best.

  • config.latest_release

    Path to the most recent deployment.

  • config.migrate?

    True if migrations are being run in this deployment, false otherwise.

  • config.node

    Various information about the current instance and other instances in the environment. See /etc/chef/dna.json on an instance for an example of what node will return. If the information you require is available via some other method, it is preferable to use the other method to get it. node is just here as a catch-all.

  • config.previous_release(current=latest_release)

    Path to the deployment prior to current. If nil, then current is the newest deployment.

  • config.oldest_release

    Path to the oldest deployment

  • config.release_dir

    Path to the directory containing the deployments

  • config.release_path

    Path to the deployment that is currently being deployed: e.g. /data/appname/releases/12345678

  • config.repo

    The URL of this application’s git repository.

  • config.repository_cache

    The path to the local clone of this application’s git repository.

  • config.revision

    The SHA of the commit currently being deployed.

  • config.shared_path

    Path to the shared dir: e.g. /data/appname/shared

  • config.stack

    The web server stack for this environment. Possible values:

    • nginx_mongrel: Nginx web server and Mongrel application server
    • nginx_passenger: Nginx web server and Passenger application server
    • nginx_unicorn: Nginx web server and Unicorn application server

Helper Methods

The following are helper methods in the deploy hook:

  • debug()

    Logs messages to STDOUT and allows you to add log-in capabilities to your deploy hooks. Only shows when verbose is true.

  • info()

    Logs messages to STDOUT and allows you to add log-in capabilities to your deploy hooks.

  • on_app_master(&block)

    Executes block only on the environment’s application master or single instance.

  • on_app_servers(&block)

    Executes block only on the environment’s application servers (master and slaves) or single instance.

  • on_app_servers_and_utilities(&block)

    Executes block only on the environment’s application servers and utility servers

  • on_utilities(*utility_names, &block)

    Executes block on utility servers whose names are included in utility_names. utility_namescan be passed as an array or as multiple arguments.

    • If called like on_utilities() { ... }, the block will be executed on all utility servers.
    • on_utilities("alpha") { ... } executes the block on all utility servers named “alpha”.
    • on_utilities("a", "b") { ... } is equivalent to on_utilities(%w[a b]) { ... }.
  • run(command)

    Executes command as a nonprivileged user (deploy by default).

  • run!(command)

    Executes command as a nonprivileged user. Aborts the deployment if it fails.
    Note: This command can only be used with the CLI and the engineyard gem 2.0. Deployment fails if you try to use run! when deploying your application from the dashboard.

  • sudo(command)

    Executes command as root.

  • sudo!(command)

    Executes command as root. Aborts the deployment if it fails.
    Note: This command can only be used with the CLI and the engineyard gem 2.0. Deployment fails if you try to use sudo! when deploying your application from the dashboard.
  • warning()

    Logs messages to STDERR and allows you to add log-in capabilities to your deploy hooks.


If you have feedback or questions about this page, add a comment below. If you need help, submit a ticket with Engine Yard Support.

 

Comments

User photo
Hugh Kelsey
45Robots

I'm using deploy hooks to stop my Resque processes before_migrate and start them up again before_restart. I just ran in to an issue where my migration failed and the deploy was halted however my workers were not ever restarted, this is somewhat obvious when you think about it but at the time wasn't expected.

Is there a deploy hook which runs on failure or runs always regarless of the outcome of the deployment like ensure in a begin/rescue block?

Thanks

July 29, 2012 02:40 PM
User photo
Evan Machnic
Engine Yard Inc.

Hi Hugh,

The only deploy hooks are the ones that are listed above and there is not one that runs on failure. I'd suggest using a conditional or something similar to check the status of the workers and then run the code if some condition is true.

If you have any questions about your application specifically, then please go ahead and put in a support ticket.

Thanks

July 31, 2012 01:15 PM
User photo
Ches Martin
Admin500

(Being too busy-lazy to dig around in code at the moment...)

 

Is there an exposed method for writing output to the `ey deploy` stream, to give informational messages about what your custom hooks are doing?

August 22, 2012 02:43 AM
User photo
Petteri Räty
experq

Ches: I have used standard error with success to output my warning messages. Maybe standard out will work as well for your information messages.

August 22, 2012 03:05 AM
User photo
Ches Martin
Admin500

Thanks Petteri, I've tried simply using `run %(echo 'message')` in the past but it doesn't seem to work. I'll try standard error but that feels wrong when I'm outputting information messages, not errors.

August 22, 2012 03:12 AM
User photo
Ches Martin
Admin500

`current_name` does not seem to work, at least not in engineyard-serverside 1.6.3: https://gist.github.com/4c81179a3545f20ad49f

Unless it's coming from method_missing somewhere, I can't see that it's defined on the configuration class in the library, and I don't see it in 2.0 either which appears to have made efforts to be a lot more explicit with method_missing. The block form with `on_utilities(*names, &blk)` would be fine, but unfortunately my staging env doesn't mirror production perfectly so I was trying `if environment == 'staging' or current_name == 'workers'`. I can use `node` instead, but please fix the docs, unless I've missed something and am wrong here...

By the way, it's still potentially being called in library code too: https://github.com/engineyard/engineyard-serverside/blob/9c15fbdbcd...

August 22, 2012 04:00 AM
User photo
Bluewater Webmaster
Bluewater Direct

I would like to get at the name of my instance, so that I can do something depending on whether I just deployed to production vs. deploying to a clone of production. 

Based on the documentation above, my initial thought was to use the current_name() method to get the name of the instance, as that method name seems appropriate, however the description of the method doesn't feel like it will give me what I want. 

I then decided that I'll need to grab the instance name from the node json.  From the example in the documentation above, It would appear that node['environment']['name'] will do the trick, however I wanted to verify that by looking at the contents of my /etc/chef/dna.json.  Unfortunately, that file is owned by root, and I can't view it as the deploy user.

Finally, based on the documentation above, it seems that I can use either the "node" variable or the node() method to access this information.  Assuming either works, having a node() method seems a bit redundant to me, and made to wonder which to use.

September 05, 2012 02:35 PM
User photo
Edward Anderson
account-6572

run! and sudo! were added in engineyard-serverside-adapter-2.0.0, which the web dashboard is not yet using. In other words, deploying can work on the command line but not in the dashboard if you use run! or sudo! in your deploy scripts. EY staff reports that this is actively being worked on.

September 10, 2012 07:55 AM
User photo
Hugh Kelsey
45Robots

Does on_app_servers_and_utilities(&block) run the app_master as well as app instances? If so is there a method to only run on app instances?

October 23, 2012 09:16 AM
User photo
Hugh Kelsey
45Robots

I think I answered my own question:

if current_role == 'app'

October 23, 2012 09:56 AM
User photo
Matt Scilipoti
LearnZillion

@Hugh Kelsey: how about on_app_servers(&block)?

February 08, 2013 07:26 AM
User photo
Matt Scilipoti
LearnZillion

Looking for information about testing these deploy steps.  In capistrano, I can make a task to restart resque and run just that task against the servers.  Once it works, I can add it to the deploy hooks.  Is there a similar scenario on EY?

February 08, 2013 07:28 AM
User photo
Evan Machnic
Engine Yard Inc.

Edward Anderson: The dashboard is now usingengineyard-serversideversion 2.x and so the new methods should work. Note that this also means that info has been changed in favor of shell.info

Matt Scilipoti: In terms of testing deploy hooks, currently the only way to do it is to deploy to an environment. This would be a good use case for EY-Local which you can find more about and request a feature at http://ey.io/ey-local

February 08, 2013 07:47 AM
User photo
Ches Martin
Admin500

Evan: Not so much specifically for testing as Matt asked about, but ad-hoc Cap tasks are one of the biggest things I miss with PaaS deployment systems these days. I know that's probably a big can of worms, and in some senses it engrains bad habits for situations where Chef automation, proper monitoring, etc. should be used, but ad-hoc tasks are still often really handy. Giving a command to `ey ssh` can sometimes fill a need. Anyway, probably better forums for this kind of feature request discussion.

Big thanks for the engineyard-local pointer though, I've bugged EY folks since forever for a vagrant box for testing Chef recipes, and I completely missed the release of this thing.

February 08, 2013 08:09 AM
User photo
William Watson
Ad-Summos

Shouldn't input_ref be listed in this document?

March 07, 2013 06:46 AM
User photo
Keri Meredith
Engine Yard Inc.

Hi William, Yes; we need to update this doc with the new variables:

environment_name: name of the environment
account_name: name of the account the environment belongs to
deployed_by: name of the Engine Yard Cloud user that triggered the deploy
input_ref: the branch name that was given as input to the deploy (instead of the commit SHA that it was reduced to for the deploy)

Thanks for the reminder. kjm

March 07, 2013 06:52 AM
User photo
Tom Hoen
GiveCorps

Can the code within a deploy hook access command line switch values? In other words, i have a deploy hook that needs to run almost every time I deploy. On the occasions when I don't want to the hook to run, I would like to pass a switch (or option, similar to --migrate) that my deploy hook could look for and exit without running. I didn't see any mention of the parameters passed to the command line "ey deploy" being accessible, but hoped they might be. 

June 04, 2013 04:14 PM
User photo
Steve Hull
RAAProduct1

Turns out that many of these are undefined when you rollback.  Broke rollback functionality for us to use them.  Suggestions?

June 04, 2013 05:00 PM
User photo
Ches Martin
Admin500

@Tom Hoen: You can use the verbose <code>--extra-deploy-hook-options</code> param -- see <code>ey help deploy</code>. Docs seem scattered on this, but you use it like <code>ey deploy --extra-deploy-hook-options=skip_foo:true</code>, and then you can access <code>@configuration[:skip_foo]</code> as a hash key in deploy hook scripts. According to <a href="https://github.com/engineyard/engineyard/pull/144">this ticket</a> it may work as <code>config[:skip_foo]</code> now, but I have old code with the ugly ivar reference that still works last I checked.

[God knows what kind of markup is supported in comments on this knowledgebase, rich-text editor with no preview function... Hoping for plain HTML]

June 05, 2013 03:46 AM
User photo
Ches Martin
Admin500

Markup fail. Engine Yard staff, please consider optimizing the comment system for a technical audience and content.

June 05, 2013 03:48 AM
User photo
Tom Hoen
GiveCorps

@Ches Martin - That will work. Thanks for the tip!

On code blocks, since EY is using the TinyMCE editor, they could add a button for the built-in "code" format.  http://www.tinymce.com/wiki.php/configuration:formats

 

June 05, 2013 06:12 AM
User photo
Keri Akin
Orchestra-keri-akin-hotmail-com

Hi, using my non-EY testing login to see how TinyMCE editor looks to customers. I see what you mean, Ches. No HTML button! (which is how I control the code/pre tags)

<code>testing a code line</code>

<pre>testing a code line</pre>

Will look into our alternatives with Zendesk. thanks, kjm

June 05, 2013 09:38 AM
User photo
Keri Meredith
Engine Yard Inc.

Logged a ticked for Zendesk so we can explore alternatives with this TinyMCE editor. thanks! kjm

[ZD-106]

June 05, 2013 09:44 AM
User photo
Keri Meredith
Engine Yard Inc.

@Tom Hoen, an additional note from the developer on your original question ...

The --config switch is for that purpose.

ey deploy --config custom:value
June 05, 2013 10:32 AM
User photo
John Yerhot
Engine Yard Inc.

@Steve Hull I updated the ticket you have open with us about the variables not being around on rollback - feel free to update the ticket if you have any questions and thanks for pointing it out.

June 17, 2013 11:55 AM
User photo
Ches Martin
Admin500

Is there a public GitHub issue regarding the undefined vars on rollback, please? You can imagine how unpleasant it is to discover that your rollback is broken when... you need to roll back.

July 09, 2013 10:16 AM
User photo
Steve Hull
RAAProduct1

@ches I don't believe there is a public GH issue regarding rollback.  And yes, I know first-hand how unpleasant it is to discover rollback is broken. Capistrano designed rollbacks in a way that should be nearly instantaneous for a good reason.  Sometimes you need to quickly abort and cutting a hot-fix branch and redeploying feels like time wasted.  I'm very familiar.  In my (private) support ticket with EY, it was explained that they basically don't support the rollback command at all and recommend that customers not try to use it.

 

To which I answer -- then take that command out of your tool.  With the command still in there, it feels like a (trollface) => (fuuuu) kind of moment when shit is hitting the fan.

July 09, 2013 11:57 AM
User photo
John Yerhot
Engine Yard Inc.

HI Ches.  Steve is right, it's not a Github issue at all.  Our documentation failed to mention that two variables, input_ref and deployed_by, do not work when using the engineyard gem's rollback command.  

I would like to clarify one part - we do support the rollback command, but it's important to understand the right use case for it and that rollback really only change a symlink to the previous release.  More information about the command is available in the Readme.

For simple deployments with no database migrations or complex deployhooks it works great to rollback a flawed release.  However, it will not fix data transformation and deploying another release that corrects the data is often the best route.  

There were a number of missteps in Steve's ticket and we left the impression that the command was not supported.  Further our documentation didn't account for the variables not being available.  Even more, it's clear we need to better document what the rollback command actually does and when it may be appropriate to use it versus deploying a corrected release.  We're working on a large overhaul of the entire deployment documentation that will cover it.  We think you'll like it.

Sorry for all the confusion guys.

John Yerhot, US Support Manager.

 

July 11, 2013 05:16 PM
User photo
Ches Martin
Admin500

Thanks for the forthcoming response John, appreciated. I do understand that rollback is something of a myth and agree that best practice should guide people toward rolling forward instead. This is thankfully not something we've needed often, but unfortunately in just such a situation that a symlink switch was an appropriate and swift course of action, we found out the hard way about this variable situation. I hope this can be addressed in a way that is less prone to catching people by surprise at the worst of times.

July 11, 2013 11:46 PM
User photo
Courtenay Gasking
ENTP - Production

config.revision is not, in fact, the current revision for us. It points to a commit a year or so old.

November 20, 2013 06:54 PM
User photo
Keri Meredith
Engine Yard Inc.

Hi Courtenay, you might want to log a Support ticket in case there's something wrong here. thanks, kjm

December 05, 2013 03:11 PM
User photo
Ryan Dowell
ValleyViewHospital

The examples in the scripts under the "Shell commands" should be updated with the new style for the API. release_path -> config.release_path, shared_path -> config.shared_path. Thank you!

March 28, 2014 10:22 AM
User photo
Keri Meredith
Engine Yard Inc.

Hi Ryan, thanks for the catch. We'll take a look! kjm

March 28, 2014 04:32 PM
User photo
Diana Lam
Engine Yard Inc.

Hi Ryan. The doc has been updated. Always appreciate the interaction. Please let us know if you see anything else.

-diana

April 01, 2014 02:08 PM
User photo
Christopher Haupt
mobirobo

Hi all,

We ran in to this recently, so this is a warning/feature request. We have some very large directories in one client's Rails project that are symlinked off of public. Currently when using the engineyard-serverside in production as of this writing, there are various places in the code that make a call into ensure_ownership, which itself checks that all files in the given path (e.g. current) have proper user and group ownership.

This behavior is killing deploy times on this project, since there are 100's of GB of files symlinked that "find" is trying to analyze and it looks like it is following those links. This method appears to be called at times that are not easily reachable to play games with removing/readding various needed symlinks.

It would be very cool to add a new callback or other config options to provide a means of filtering what is checked.

Is there a good way to contribute patches or does anyone have a suggestion (other than "don't put all that content there" :) )

-Chris

April 01, 2014 05:56 PM
User photo
Ches Martin
Admin500

@Chris, not to say your feature request might not be a valid one, and apologies for what may sound like a troll, but seriously: don't put all that content there :-)  If you ever need to scale the app horizontally to additional app/web server instances, you'll face the pain of needing to share those files. NFS is fiddly to set up, adds a point of failure, and often doesn't perform/scale very well in a web app workload. IMO your team should strongly consider migrating those files to S3 or the like.

Sorry to veer a bit off-topic -- I bet Engine Yard staff can follow up privately with some advice for that situation.

April 03, 2014 02:24 AM
User photo
Tasha Drew
Engine Yard Inc.

Hi Chris and Ches, 

Thanks for the note! I'll check with our serverside specialist and get back to you.

Thanks, Tasha

April 03, 2014 07:18 AM
User photo
Christopher Haupt
mobirobo

@ches: No problem. For background for folks who may be in similar situations: This project is a migration for a (very) long-time EY customer who is on Terremark infrastructure that currently relies on NFS for hosting particular kinds of content. We are transitioning the app to AWS and rewriting the portions of the app that had file system dependencies. Since this is such a mission critical app, we are doing things in stages, which include moving new content to S3 and then migrating the legacy content to S3. For initial staging and development testing, we have the large data set locally on EBS volumes. Hence the long deployment times. Once in production, all will be on S3 or similar cloud storage.

For all: The solution for this was handled in a private ticket, but the approach (since I completely forgot about this and I didn't rewalk the doc): using the advanced customization override files (rather than deploy hooks): https://support.cloud.engineyard.com/entries/20996661-Customize-You... 

This allows us to patch the behavior I mentioned above and simply filter specific directories out of the scan. The deployment time went from 30 minutes to 15 seconds once I pushed that live.

April 03, 2014 08:33 AM
User photo
Martin Emde
Engine Yard Inc.

Here is a rough draft of the solution we offered Christopher Haupt.

requirement = Gem::Requirement.create("~>2.3.0")
unless requirement.satisfied_by?(About.version)
raise "engineyard-serverside version change detected. Make sure this patch works before deploying to production"end

# In this example, the command below is not altered.
# Here is where you would make your modification to the chown command.
# Methods in the kernel scope here are eval'd into the Deploy object.
# This will then override all calls to ensure_ownership.
def ensure_ownership(*targets)
  sudo %|find #{targets.join(' ')} \\( -not -user #{config.user} -or -not -group #{config.group} \\) -exec chown #{config.user}:#{config.group} "{}" +|
end

Usually I don't recommend using eydeploy.rb files because of the hassle of maintaining compatibility with each version. I think this case warrants using it.

The deciding factors for using eydeploy.rb files are that the change is something that is too specific to add a ey.yml configuration option. An option for this would usually cause more problems for customers than it would solve, since altered permissions frequently cause problems. Also, this change is intended to temporarily fix something as they migrate and the customer would rather maintain the deploy consistency themselves.

The version check at the top is a very useful feature in all eydeploy.rb files. Maintaining this file is up to the customer, but we always do our best to leave internals unchanged in following SemVer. Minor and major version changes can and will significantly alter the internal methods that eydeploy.rb hooks into.

April 03, 2014 11:53 AM