Engine Yard Developer Center

Engine Yard Release Notes - February 2015

The updates described are either important (where you need to take action) or of interest (you might want to know about these changes but you don't need to do anything). 

Hotfix: Engine Yard Gentoo 2009 stack upgrade

February 10th, 2015

Action: We recommend you test this hotfix in your staging environment as soon as possible; then, when that is validated, click the Upgrade button for your production environment.

Note: For clarity, since we now have 2 Gentoo stacks, we refer to this stack as the Engine Yard Gentoo 2009 stack. You can access it by using the Stack select field in the Environment UI:

stack_stable-v2.png

It's best practice to upgrade your Engine Yard Gentoo 2009 (stable-v2) stack regularly for the latest security and product updates. This week's updates:

  • Upgrades MySQL 5.6 to 5.6.22: See upgrade instructions for minor database version changes.
  • Uses an upgraded base disk image that contains kernel and package updates, including the new glibc to resolve the "Ghost" vulnerability.

Update instructions to Resolve Ghost Vulnerability

This release uses a base disk image that includes the glibc version that has been patched against the "Ghost" security vulnerability identified in CVE-2015-0235.  This will ensure all new instances are not vulnerable.  However, this will not purge the vulnerability from your running environment(s). Once the upgrade process is complete, perform one of the following (listed in recommended preference order):

  • Simple, quick, with some downtime:
    Terminate and rebuild the environment using the most recent snapshot to fully purge the vulnerable software from all involved instances.
  • Maximized uptime:
    1. Cycle Application slave and utility instances (add new instance, then remove the older one, for each existing instance)
    2. Promote one Application slave to master
    3. Remove all existing DB slaves, making note of any names
    4. Add new DB slave, and wait for it to catch up to master
    5. Promote DB slave to master (this is the only step that results in downtime)
    6. Add back in DB slaves, using the previous names if applicable

NOTE: This requires a cluster setup -- those using single server setup will need to use one of the other methods.

  • Where speed is the most important:
    Disable takeover functionality, then manually reboot all instances simultaneously within the environment. 
    1. Schedule a time where a short duration of downtime can be tolerated
    2. Edit environment and change the Takeover Preference to Disabled
    3. Save the environment change, then click Apply.
    4. Wait for Apply to complete.
    5. Log into each instance and reboot simultaneously: sudo shutdown -r now
    6. Wait for the instances to reboot and confirm your app is working
    7. If there are any issues after the reboot, click apply and re-deploy your app
    8. Open a support ticket if you need further assistance
    9. Finally, edit the environment to restore your Takeover Preference to its previous value, save, then click Apply.
  • If absolute minimum downtime required and you're fully comfortable with linux process management:
    Manually kill and restart selective processes running the old software by checking for them using:
    lsof | grep DEL.*lib.*\.so | awk '{print $2, $1}' | sort -un

We understand these options are delicate to your operations. If you have questions, issues, or are not comfortable performing the above steps then please let us know via a support request. Our Application Support engineers are available to assist however possible.

For more information on Engine Yard Gentoo 2009, see the Engine Yard Gentoo 2009 docs.


Hotfix: Engine Yard Gentoo 12.11 stack upgrade

February 2nd, 2015

Action: We recommend you test this hotfix in your staging environment as soon as possible; then, when that is validated, click the Upgrade button for your production environment.

Note: For clarity, since we now have 2 Gentoo stacks, we refer to this stack as the Engine Yard Gentoo 12.11 stack. You can access it by using the Stack select field in the Environment UI:

stack_stable-v4.png

This release installs a glibc version that has been patched against the "Ghost" security vulnerability identified in CVE-2015-0235.  However, this will not purge the vulnerability from your environment(s). Once the upgrade process is complete, perform one of the following (listed in recommended preference order):

  • Simple, quick, with some downtime:
    Terminate and rebuild the environment using the most recent snapshot to fully purge the vulnerable software from all involved instances.
  • Maximized uptime:
    1. Cycle Application slave and utility instances (add new instance, then remove the older one, for each existing instance)
    2. Promote one Application slave to master
    3. Remove all existing DB slaves, making note of any names
    4. Add new DB slave, and wait for it to catch up to master
    5. Promote DB slave to master (this is the only step that results in downtime)
    6. Add back in DB slaves, using the previous names if applicable

NOTE: This requires a cluster setup -- those using single server setup will need to use one of the other methods.

  • Where speed is the most important:
    Disable takeover functionality, then manually reboot all instances simultaneously within the environment. 
    1. Schedule a time where a short duration of downtime can be tolerated
    2. Edit environment and change the Takeover Preference to Disabled
    3. Save the environment change, then click Apply.
    4. Wait for Apply to complete.
    5. Log into each instance and reboot simultaneously: sudo shutdown -r now
    6. Wait for the instances to reboot and confirm your app is working
    7. If there are any issues after the reboot, click apply and re-deploy your app
    8. Open a support ticket if you need further assistance
    9. Finally, edit the environment to restore your Takeover Preference to its previous value, save, then click Apply.
  • If absolute minimum downtime required and you're fully comfortable with linux process management:
    Manually kill and restart selective processes running the old software by checking for them using:
    lsof | grep DEL.*lib.*\.so | awk '{print $2, $1}' | sort -un

We understand these options are delicate to your operations. If you have questions, issues, or are not comfortable performing the above steps then please let us know via a support request. Our Application Support engineers are available to assist however possible.

For more information on Engine Yard Gentoo 12.11, see the Engine Yard Gentoo 12.11 docs.


If you have feedback or questions about this page, add a comment below. If you need help, submit a ticket with Engine Yard Support.

Was this article helpful?
1 out of 1 found this helpful
Have more questions? Submit a request

Comments

Please sign in to leave a comment.

Powered by Zendesk