Knowledge Base/Engine Yard Support Tools/Monitoring Tools

PagerDuty Guide for Premium Support Customers

Severin Discher
posted this on June 06, 2012 02:31 PM

The PagerDuty notification tool is free for Premium Support customers and their teams, limit three (3) users.  Need more?  Let us know via ticket!  

Benefits of PagerDuty:

  • NEW!  Integrate your own services to PagerDuty, such as New Relic or Nagios, to broaden your alert system. 
  • Prompt and reliable notifications for ticket events of your choosing.
  • Automated communication, which can save you precious minutes during an emergency.
  • Deeply customizable on-call and escalation settings for everything your team needs.

Consider which types of event you want PagerDuty to contact you about. Be sure to let us know what you decide so we can configure our portion! Here are some examples of acceptable events:

  • Notify me when Engine Yard creates a proactive ticket.
  • Notify me when an Urgent or High priority ticket is created.
  • Notify me when a ticket is created which mentions "outage" or "AWS."  

Note: PagerDuty is a notification system for you and your team to maintain. Engine Yard staff may have an impact on the initial ticket that begins the notification process, but otherwise is uninvolved with this notification system.

To add users to PagerDuty

  1. Login to PagerDuty at http://pagerduty.com. You should have already received your login credentials from an Engine Yard ticket.

    The PagerDuty Dashboard appears.

    1.png 

  2. Click Add Users to begin populating your PagerDuty list of contacts.

  3. Click Invite New Team Member, and add your users.

    Your PagerDuty service permits up to three users including the one you are currently using. However, our pagerduty@engineyard.com account does not count as one of the three. The limit is not tied to the available roles (Admin and User); for example, you can have three administrators and no users, or one administrator and two users. 

    The Admin role can create and delete users, modify contact details, and schedule.

  4. Add each user’s contact information.

    User settings are on the Users tab. Click the applicable user account.

    pd_profile.png

  5. Add available cellphone number(s) to the SMS and/or Phone field. Adjust the Notification rules as desired. PagerDuty follows these rules when it notifies the individual user of an event.

    Note: Best practice is to use PagerDuty to send SMS/Phone notifications instead of email. This is because most events already trigger an email from our helpdesk system. 

  6. (Optional): If your team takes turns being on-call, consider setting an On-call schedule via the On-Call Schedules tab at top. Alternatively, you can make manual changes by logging into the PagerDuty interface later when you’re ready to adjust the process.

    pd_oncallFIXT.png

    The screenshot shows a balanced on-call schedule between Larry and the boss, alternating each day of the week. To give Larry a larger shift than his boss or even overlapping weekly coverage, click Add Another Layer and move one user to it.

  7. Create an Escalation policy: (Escalation policies tell PagerDuty what order your users should be notified in).

    Click the Escalation Policies tab at top, and then click the green Create Escalation Policy button at the right.

    pd_escalationpolicyFIXT.png

    Note: The above configuration example supersedes the need for an on-call schedule.

  8. Add a policy name (Default is fine), and adjust and add rules as needed.

    Tip:If you’re unsure what to set on this page, ask yourself “Do I want the same person(s) notified during an event, or will they alternate?”

  9. Important! Before user creation is complete, each new member must click the activation link sent to their mailbox from PagerDuty.

After you get confirmation that an Engine Yard engineer has configured our portion, PagerDuty should be ready to notify your team.

Important! We recommend that you test this configuration before relying on it. Depending on what criteria you’ve requested from us, you can either create a test ticket or ask an Engine Yard engineer to do it for you.

How escalation works

When you or your team receives a PagerDuty notification of an event, it needs to be resolved or else the service will proceed with the rest of your escalation procedure on the assumption that the first person is unavailable. To resolve an event, follow the instructions sent via SMS/Phone.

Below is an example SMS exchange with PagerDuty:

6.png