Manage Snapshots

Updated:

Updated: July 10th, 2014

Snapshots are an important part of a high availability strategy but keeping lots of old snapshots around can add up in usage charges. If you find that you have a number of snapshots that you no longer need, you can use the snapshot management / delete function to clear them out and reduce your monthly costs.

Snapshot management allows you to set the snapshot policy at the account level and at the environment level; you can also delete unneeded snapshots from the account or environment levels.

Tip: Too-frequent snapshots or backups can cause disk I/O performance issues. If you need rapid failover, consider setting up a database replica.

Get started with snapshot retention policy on Engine Yard

Prerequisites

It's important for you to understand these aspects of Engine Yard snapshot retention policy:

  • The configuration setting for the number of snapshots applies to each instance in an environment.
  • The default snapshot retention period is set to 90 days.
  • Engine Yard deletes any snapshot that is older than the retention period.
  • You can change the snapshot retention period at the account level as well as the environment level; the policy is not inherited from account to environment.
  • The account level snapshot retention policy is used only for "orphaned" snapshots, which is somewhat rare.
  • Seasonal (periodically dormant) environments should use a retention setting that keeps at least one snapshot between seasons (for example use the 365 day or the "forever" setting).

Important: You should verify the snapshot retention period for your seasonal application environments.

Manage snapshots

You can manage snapshots at the account level and the environment level.

To manage snapshot retention policy at the account level

  1. In Engine Yard, click Tools > Snapshots.

    The Snapshot Management page appears, showing the snapshot retention period for the account level (used for "orphaned" snapshots only).

    All snapshots across all the account's environments appear in the snapshot list.

  2. Continue with Step 4 below.

To manage snapshot retention policy at the environment level

Important: We recommend testing in a staging environment before applying changes in a production environment.

  1. In Engine Yard, click Tools > Dashboard.
  2. Select an environment by clicking the environment name.
  3. On the Environment page, click Snapshots.

    snap_view_manage.png

    The Snapshot Management page appears, showing the snapshot retention period for the selected environment.

    All snapshots specific to this environment appear in the snapshot list.

  4. (Optional) Change the snapshot retention period if you need to decrease or increase the amount of time (in days) that snapshots are kept.

    snap_management.png

  5. Click Update snapshot settings.

    From this point forward, Engine Yard will delete snapshots after the specified number of days has passed for all snapshots related to any instances.

  6. (Optional) If you need to delete snapshots, select the Destroy checkbox for those snapshots.

    snap_check_destroy.png

  7. Click snap_delete_button.png

    Engine Yard immediately begins deleting the selected snapshots and displays [destroying] at the end of the snapshot description.

    snap_destroying_inprog.png

Configure snapshot retention policy

When you set up your environment, you specify the number of snapshots and the number of days to keep the snapshots.

Important: We recommend testing in a staging environment before applying changes in a production environment.

To configure snapshots in your environment

  1. In Engine Yard, click Tools > Dashboard.
  2. Select an environment by clicking the environment name.
  3. On the Environment page, click Edit Environment.
  4. Update the snapshot retention policy as needed:

    enviro_snap_settings.png

    • Number of snapshots to keep, per instance.

      Note: This applies to each instance in an environment. For example, if you have 5 instances and set this number of snapshots to 10, then your environment will accumulate 5x10 snapshots.

    • Snapshot retention period in days, for this environment. Engine Yard will delete snapshots after the specified number of days for all snapshots related to any instances associated with this environment.

      Note: This setting takes precedence; once the retention period has passed, the snapshots get deleted, regardless of the number of snapshots.

    • Snapshot frequency is based on the frequency of backups.

      Tip: You can set the frequency of backups under the Backups heading in the UI.

  5. Click Update Environment to save the new snapshot settings.
  6. Click Apply to push the policy changes to the instances.

Verify snapshot retention policy

Here are some ways that you can verify the snapshot retention policy for your environment.

To verify snapshot retention policy

  1. You can view the account-wide setting for snapshot retention period (used for "orphaned" snapshots only) using Tools > Snapshots.
  2. You can view the environment-specific setting for number of snapshots (per instance) and retention period (per environment) on the Edit Environment page. You can also view the retention period, in days, on the Snapshot Management page.
  3. You can view the date and time a snapshot was created (and therefore, when you can expect it to be deleted), on the Snapshot Management page.
  4. Once you have deleted snapshots, you can refresh the Snapshot Management page to see the remaining snapshots.
  5. You can verify the frequency of snapshots based on the frequency of backups.

Notes:

  • Deleting an instance does not make the related snapshots go away; Engine Yard deletes snapshots based on the set retention period for the environment (default of 90 days).
  • Deleting an environment does make the related snapshots go away; Engine Yard deletes snapshots immediately. In rare cases, some snapshots will not be caught during this delete process; those "orphaned" snapshots will be deleted based on the set retention period for the environment (default of 90 days).

FAQs

You might have these questions about the snapshot retention policy.

Why do I have WAY more snapshots than I expected (for example, I have 5 instances, yet have more than 500 (5x100) snapshots)?

This can happen if you cycle your instances often. Remember that the number of snapshots is per instance. So if you have 5 running instances and 10 deleted instances, you can have (5+10)x100 snapshots associated with that environment.

Why do I still have snapshots even though the related instance has been deleted?

Snapshots are kept even after an instance is stopped. This is a precaution, in case you need the snapshots to rebuild an instance. The default is to keep snapshots 90 days.

Why is there no name in the Environment column associated with snapshots in the list of snapshots for this environment?

Snapshots that appear in your snapshot management list without an associated environment name are probably "orphaned" snapshots from a deleted environment. If you no longer need them, it is OK to delete these snapshots. Or, if you do nothing, Engine Yard will delete them when the snapshot retention period (default 90 days) has passed.

Why am I being charged for snapshots even though the environment is long gone?

See above answer. The snapshot retention policy should be set based on your app's needs. If you have old snapshots that you no longer need, see Manage snapshots in an environment for information on deleting unused snapshots (even those no longer associated with a particular environment).

Can I set a retention period per instance? or per app?

No; retention period (in days) is set per environment. The account-level default is 90 days.

Can I keep more snapshots for my database (for example, 20 for database and 5 for app master)?

No; the number of snapshots setting is per instance, across the environment. Remember that snapshots are generally small and incremental and are relatively low-cost. Use backups to ensure data integrity.

What's the difference between a backup and a snapshot again?

See types of backups. See also environment options.

More information

This table provides other resources related to snapshot retention policy.

For more information about...See...
Taking snapshots Take Snapshots
Backups Back Up the Database
Creating a new environment Create an Environment
Adding instances to an environment Add Instances to an Environment

If you have feedback or questions about this page, add a comment below. If you need help, submit a ticket with Engine Yard Support.

Was this article helpful?
1 out of 1 found this helpful
Have more questions? Submit a request

Comments

  • Avatar
    Petteri Räty

    This doc says: Note: Deleting an instance or an environment does not make the related snapshots go away; Engine Yard deletes snapshots based on the set retention period (default 90 days).

    However, when trying to delete an environment you get a dialog saying:

    "Are you sure? This will delete all volumes and snapshots associated with this xyz environment. There is no undo. This may take a minute so be patient"

  • Avatar
    Keri Meredith

    Hi Petteri, Thanks for this update. We recently changed the snapshot management logic (with this recent feature GA). Looks like we need to update that dialog message to match. kjm

    [Triage-6502]

  • Avatar
    Andrew Rogoff

    How often are snapshots taken? Once a day?

  • Avatar
    Paul Tingle

    Hi Andrew, the snapshot frequency is set to daily (at 1am) by default. However, the snapshot frequency mirrors the database backup frequency selected in the environment settings, so increasing this frequency will also result in snapshots being taken at the same frequency.

  • Avatar
    Andrew Rogoff

    So, if I have the following settings:

    Database backups: every 1 hour

    Snapshots:

    - Keep the 50 most recent snapshots

    - Keep for 90 days

    It will take a snapshot every hour? Keep the last 50 - meaning it will keep snapshots for the last 50 hours? And keep them for 90 days? So, will I end up with a lot more than 50 snapshots or not? If it's only keeping the last 50 hours of snapshots then it's not keeping them for 90 days. Sorry, I'm a bit confused.

  • Avatar
    J. Austin Hughey

    Hey Andrew,

    No problem, let me try to explain.

    The 'Keep the 50 most recent snapshots' option will ensure that the 50 most recent snapshots for each of the instances is kept. So in your case this will mean that 50 snapshots for each of the 4 instances in the environment are stored, the oldest being 50 hours old. As the snapshots are incremental, each snapshot is only the size of the changes since the previous snapshot, so you are not looking at hugely increased storage costs to increase the amount of snapshots stored, if this is required.

    The 'Keep for 90 days' option is the retention period, meaning that any snapshots older than 90 days are deleted. Should any instance be deleted, or the environment shut down, the associated snapshots will be kept for 90 days, then automatically deleted, so you would not be able to reboot instances from these snapshots after this period had elapsed. (Deleting an environment would also delete all snapshots for that environment). 

    One other thing I want to bring to your attention here: snapshotting or backing up your database every hour is very likely to cause some disk I/O issues, so I strongly advise against it. Your performance could really suffer. Instead, if you need that kind of fast failover granularity or want to minimize data loss in terms of downtime, really the right way to go is adding a database replica, which is real easy with the dashboard (both MySQL and PostgreSQL are supported).

    Hit us up with more questions if you have them. Hope this helps.

    P.S. At this time the snapshots and backups are still running at 2 hour intervals on your environment, as you have not yet clicked 'Apply' to push this change out to the instances.

  • Avatar
    Andrew Rogoff

    OK, thanks - yeah, that's really helpful. Maybe you should change the language slightly. Maybe it should read something like:

    • Keep the [dropdown] most recent snapshots and delete each one after either [dropdown] days or to make room for new snapshots

    I presume that's what would happen? That if you're backing up every hour and keep 50 snapshots then they would never get a chance to be anywhere 90 days old (unless the instance was deleted)? So the "keep for x days" actually becomes redundant in a lot of cases (unless the instance is deleted)?

    One other thing, it would be really useful if your system could retain backups for much longer periods. At the moment, if you back up every 2 hours, your oldest backup is only going to be 17 days old (2 x 200 covers a 400 hour period). So, you can never go back and look at a backup from 1, 2 or 3 months ago?

    Thanks for the advice on the replica - I'll look into that.

Please sign in to leave a comment.