Every Engine Yard Cloud or Managed account comes with complimentary New Relic standard account and we highly encourage every customer take advantage of it.   Here at Engine Yard Support, we use New Relic on a daily basis to help diagnose everything from misbehaving instances to request queuing issues.

To get started, be sure to check out our internal documentation for installing and configuring the New Relic add-on for your apps.

Once you’ve installed, configured, and can access New Relic you’re ready to get started.

New Relic Best Practices

1. Request Queuing - Understand it!

The first place I look in New Relic is always the App Server Overview page.  Nothing gives a better high level picture of an application’s status better.

A commonly confusing metric is time spent in Request Queuing.  Not only does it sound nasty (no one likes the word queuing!) but it’s also not well understood what it means.

Request queuing is the time from when the X-Queue-Start header is inserted into the request (by Nginx on Engine Yard Cloud) and when ActionController is hit for the first time.  You heard right - time spent in HAProxy is not included.  This also means that time spent in Rack Middleware is included since it is before ActionControl.  However time spent there can be tracked as well by inserting X-Middleware-Start at the beginning of your Middleware.  Simply add this to the beginning of your Rack Middleware:

env["HTTP_X_MIDDLEWARE_START"] = "t=#{(Time.now.to_f * 1000000).to_i}"

 

However, Middleware metrics will be available only in custom dashboards which is not available in the Standard account provided free with Engine Yard Cloud accounts.  Upgrading to a Pro account and creating a custom dashboard on the "Middleware/all" metric will show time spent in middleware.

 

Request queuing is usually a symptom of a larger problem within the application which sometimes can be temporarily remedied by adding more application servers.  In the end, addressing the areas of your application that are slow will yield the biggest bang for the buck since your Passenger and Unicorn workers will spend less time on each request.  The faster workers can churn out requests, the smaller the request backlog will be and the fewer servers you will need.

2. Dig deep!

In order to diagnose things like Request Queuing, New Relic make it easy to see exactly what is slow in your application.  The Web Transactions page is great for letting you know the biggest offenders.

But, you can drill down even deeper by clicking on the action that is slow and using the Performance Breakdown tab.  Now you can see exactly what is taking so long!  Maybe it’s some slow view rendering or maybe it’s just some slow SQL - know you know!

1.  Extend New Relic!
   

New Relic supports quite a bit of instrumentation out of the box, but there is even more you can easily keep track of.  

You will definitely want to check out the rpm_contrib gem to gain some instant instrumentation for all the other components you use - Resque, MongoDB, Sphinx, and tons more.   It also adds out of the box support for other frameworks like Camping and Sinatra.   It doesn’t however include instrumentation for Redis, but  Evan Phoenix's Redis instrumentation Gem fills that void quite nicely.

But why stop there?  You can write your own custom instrumentation using New Relic’s Agent API.

1. New Relic isn't just for your application!

The New Relic account that comes free with an Engine Yard account also includes detailed monitoring of your Engine Yard instance's health.  Simply enabling New Relic monitoring in for your environment will install, configure, and run the New Relic Agent.  Then just make sure newrelic_rpm to your Gemfile and New Relic will collect detailed information on your instances: CPU, memory, disk, network, and processes.

How about you?

How do you use New Relic?  What feature do you use most often?  Do you have any custom instrumentation you’d like to share?  We would love to know your best practices.