Knowledge Base/Community Forums/Best Practices

Using Memcached on Engine Yard Cloud

Jim Neath
posted this on November 09, 2012 12:59 PM

Updated: April 22nd, 2013

What is memcached

Memcached is a high-performance, in-memory key-value store that can be used to speed up your Rails applications by caching the results of database calls, API calls or page rendering.

Using memcached in a clustered environment

Rails 3

The memcached client that we recommend for use with Rails 3 is Dalli by Mike Perham, the former maintainer of the memcache-client gem.

The first step is to add dalli to your Gemfile:

gem 'dalli' 

By default, EY Cloud sets up and monitors memcached on solo, app master and app instances. We also write a memcached config file, memcached.yml, to your application's shared config directory, /data/yourapp/shared/config/. This file will be symlinked to /data/yourapp/current/config/memcached.yml when you deploy your application.

Finally, we need to parse the memcached.yml and tell Dalli where our memcached servers are located. We can do this by adding the following to config/environments/production.rb:

# parse the memcached.yml
memcached_config = YAML.load_file(Rails.root.join('config/memcached.yml'))
memcached_hosts = memcached_config['defaults']['servers']
# pass the servers to dalli setup
config.cache_store = :dalli_store, *memcached_hosts

When app instances are added or removed, your memcached.yml will be updated automatically. So your application will always be using the correct hosts.

Rails 2.3

Unfortunately, Dalli offers no support for Rails 2.3 applications. However, Rails 2.3 comes with a prebundled version of the memcache-client. We simply need to make the following updates to config/environments/production.rb:

# parse the memcached.yml
memcached_config = YAML.load_file(Rails.root.join('config/memcached.yml'))
memcached_hosts = memcached_config['defaults']['servers']
# pass the servers to memcached setup
config.action_controller.cache_store = :mem_cache_store, *memcached_hosts

Using a utility instance as your memcached server

To run memcached on a utility instance, first of all, follow the steps listed above to setup your cache store to use the hosts listed in memcached.yml.

Next, you will have to use a custom Chef recipe to set up memcached to run on your utility instance. The following is an example of a Chef recipe that will set up memcached on a utility instance named memcached. It will then write a new version ofmemcached.yml that points to the utility instance.

https://github.com/jimneath/ey-memcached-util


If you have feedback or questions about this page, add a comment below. If you need help, submit a ticket with Engine Yard Support.

 

Comments

User photo
Rien Swagerman
viewbook

So, the default YML file by EY looks like this:

defaults:
ttl: 1800
readonly: false
urlencode: false
c_threshold: 10000
compression: true
debug: false
namespace: ""
sessions: true
fragments: true
memory: 128
servers:
- ip-xx-xx-xx-xx.ec2.internal:11211
benchmarking: true
raise_errors: true
fast_hash: false
fastest_hash: false

production:
benchmarking: false

 

However Dalli does not pickup any values from yml file except for the servers. Dalli does not support yaml configuration.

It looks like the config options in the default EY memcached.yml originate from the config file from 'Cache Fu'?

Just to clarify...

 

November 20, 2012 02:44 AM
User photo
Dave Krupinski
Laika

Looks like the Rails 3 example has an error.  Since the servers are actually stored under 'defaults' , it should not be fetching the current environment's values from the hash.

Corrected:

# parse the memcached.yml
memcached_config = YAML.load_file(Rails.root.join('config/memcached.yml'))
memcached_hosts = memcached_config['defaults']['servers']     
December 04, 2012 04:46 PM
User photo
Duarte Henriques
SeedrsApp

Here is an alternative:

 

  memcached_hosts = YAML.load_file("#{Rails.root}/config/memcached.yml")[Rails.env]["servers"] rescue []
  config.cache_store = :dalli_store, *memcached_hosts

 

This help page should be updated with something that works.

March 19, 2013 12:13 PM
User photo
Tasha Drew
Engine Yard Inc.

Thanks for the feedback on this Duarte, Dave, and Rien. I'll pass this on to our technical writer so it can get updated.

March 19, 2013 01:06 PM
User photo
Chintan Doshi
account-6388

I think there is still an error for Rails 3 example. Is the [Rails.env] part required here ?

memcached_config = YAML.load_file(Rails.root.join('config/memcached.yml'))[Rails.env]
April 10, 2013 06:17 AM
User photo
Evan Machnic
Engine Yard Inc.

Hi Chintan,

Thanks for pointing this out. You are correct that the [Rails.env] part is not needed here. We will make sure to update the documentation to reflect this.

Thanks,
Evan Machnic

April 10, 2013 06:36 AM
User photo
Keri Meredith
Engine Yard Inc.

Code samples updated. Thanks all for the feedback. Apologies for the delay. kjm

April 22, 2013 01:25 PM
User photo
Matt Scilipoti
LearnZillion

Thanks for this page, it allowed me to get up and running with memcached quite quickly (and thanks for the updates).  Could you add a little discussion for why we would use the utility instance instead of the existing, default memcached instances on each server?  Moving back to a single point of failure concerns me, but I can see a benefit of not sharing the memory with the apps.  Are their other advantages/disadvantages?

July 29, 2013 05:35 PM
User photo
Kevin Rutten
Engine Yard Inc.

Hi Matt,

Cloud instances can be added or removed on demand (and might be replaced if they start having issues). The IP's related to instances will change which will change the list in memcache.yml (by default a list of all App instances).  When the list changes the hashes Memcache will use will change invalidating the cache.  Using a utility instance allows you to dedicate more memory to Memcache and will be more stable (IP's change less often).

Regards,
Kevin

July 29, 2013 06:59 PM
User photo
Doug Schlenker
account-4930

If you are using a separate instance, the recipe linked in the article and using the stable-v4 stack change the version to:

memcached_version = '1.4.5-r1'
August 13, 2013 03:48 PM
User photo
Brian Kilgore
Intensity

I'm new to EY and I'm trying to deploy my PHP app with memcached support. I see that the memcached server is provisioned when I boot my environment, and I see the configuration details in config/memcached.yml, but I don't see how to access those settings and point my app to the correct memcached server address. I've tried reading the yaml file directly in my code and extracting the info, but haven't had any success. Any help would be greatly appreciated.

November 18, 2013 01:13 PM
User photo
Tasha Drew
Engine Yard Inc.

Hi Brian!

I've opened a ticket for you so our support team can help you with your question. 

Cheers, Tasha 

November 18, 2013 01:32 PM