Knowledge Base/Community Forums/Custom Components

Using Redis

James Paterni
posted this on May 01, 2012 02:12 PM

Updated: September 10th, 2013


Redis is an open source data structure server which can store strings, hashes, lists, sets and sorted sets in memory.

Redis was written in C and stores data in-memory in order to make it quick and responsive.  The developers of Redis have taken special care to insure that the server is as responsive as possible at all times.  For instance, all database persistence is done in a separate forked process to ensure slow disk i/o does not impact Redis’ in-memory speed.

Although Redis is an in-memory store it does allow 2 methods of persistence. Periodic snapshots can be taken of the database at user defined times, or each and every update to the database can be recorded to an “Append only file”.

Which persistence method to use


You should always at least enable snapshots of your database.  Since redis performs all persistence in a separately forked process, the snapshots and updating of AOF (append only file) should not impact the speed of the database.

Which persistence method to use depends on how important your data is.   If you can afford to lose a few minutes worth of data then simply using the database dump (bgsave) method is preferred, however, if your data is very important and you can’t afford to lose a single minutes worth of data, you will want to use both the dump (bgsave) and AOF methods.  Since Redis 2.4, redis will ensure that AOF and bgsave commands do not run at the same time preventing any extra I/O so there is no reason not to run both.

Database Dumps


Previously I mentioned that Redis will create a thread and dump the database in the background.  Well, I lied.  It actually depends on how you configure Redis.

Configuring Redis to dump requires 2 parameters.  One specifying how frequently to dump and the 2nd telling Redis to only dump if X many keys have been modified.   

For example, ‘save 60 10’ will save a new dump of the database every 60 seconds, but only if at least 10 keys have changed since the last dump.  This helps prevent unneeded dumps from causing extra I/O on your instance.

You can make multiple save statements in the config file for various cases.  

The AOF File


The append on write file is great if your data integrity is important, but it’s also good to use ‘bgsave’ to make restarting redis quicker, as well as making migrations easier.  Whenever using AOF it’s suggested to also use bgsave.

If unmonitored the AOF file can grow indefinitely.  You should always configure the AOF file to automatically rewrite itself to be as small as possible.  To do so you need to configure the auto-aof-rewrite-percentage, and auto-aof-rewrite-min-size options in your redis.conf file.

Redis will automatically rebuild the AOF file to the smallest size needed to rebuild the current database if the percentage of change in writes is greater than “auto-aof-rewrite-percentage” and the AOF file is larger then “auto-aof-rewrite-min-size” options.

The github.com/engineyard/ey-clooud-recipes/redis example sets auto-aof-rewrite-percentage to 100 and the auto-aof-rewrite-min-size to 64MB.

Security


Redis has little built in security.  The data sent between clients is not encrypted, and redis normally accepts any connections.  This type of configuration is OK as long as the instance Redis is running on is secluded from the outside world.  

Redis does have an authentication method, but due to the lack of encryption it is best to secure Redis on the host side.
When hosting Redis on a utility slice at Engine Yard, the default security groups prevent outside traffic from hitting your redis server and requires no further configuration.

Backups


Backing up your redis db is as simple as making a copy of your database dump to a secure location.

If using the example at github.com/engineyard/ey-cloud-recipes/redis Redis will be configured to store the dump file in /data/redis/ which will have periodic snapshots taken automatically, depending on how you’ve configured your environment.

Redis At Engine Yard


The best way to use redis on app cloud is to put Redis up on a utility instance.   Since Redis shouldn’t run a database larger than the available memory, be sure to select an instance size that will allow your database to grow.

Enabling Redis on your environment can be as simple as enabling the redis recipe located at http://github.com/engineyard/ey-cloud-recipes/tree/master/cookbooks/redis, and creating a utility instance named Redis.

If you need to use the AOF file for better persistence you will need to further modify the template and change ‘appendonly no’ to ‘appendonly yes’.  Automatic rewriting of the AOF file will already be enabled.

For more information on the example redis recipe, see https://github.com/engineyard/ey-cloud-recipes/tree/master/cookbooks/redis/README.md.

 

Comments

User photo
Pernod Ricard
wispr

The recipes links doesn't seem to work. Could you please update this article?

Kind regards,
Nicklas 

September 21, 2012 12:26 AM
User photo
Tyler Bird
Engine Yard Inc.

Pernod,

We'll get that updated soon.  In the meantime here's the corrected link:

https://github.com/engineyard/ey-cloud-recipes/tree/master/cookbooks/redis

Thanks for letting us know!

September 21, 2012 08:15 AM
User photo
James Paterni
Engine Yard Inc.

Pernod,

Sorry about that.  The links have been updated. 

Please let me know if you see any other issues with this article.

 

Thanks!

James Paterni

September 21, 2012 09:33 AM