Redis Setup Drupal 7 on Elastic Beanstalk with ElasticCache

By Geff Harper on June 21, 2018

Tagged in: , , ,

The usual redis connection with drupal 7/8 is quite easy. Installing the phpredis extension and redis drupal module should be enough for most cases.

But when it turns to cloud platform, a lot of extra settings will need to be done to get the redis connection working properly.

Install the phpredis extension

The default elastic beanstalk (EB) platform is AWS’ OS. So the PECL is not installed by default when you deploy the php environment. It just makes installing Pear extensions even more difficult. Fortunately, the EB comes with .ebextensions configuration files you can use to do custom config for your EB servers. To add the custom config, you will need to add a .ebextensions/ directory at the same level as your index.php root. For example, you put all your code including index.php in the directory html/, then the directory .ebextensions/ will be on the same level as html/. The structure will looks like:

.ebextensions/
html/index.php

Then create a file “php-modules.config” with in the .ebextensions/ folder and add the following content:

# these commands run before the application and web server are
# set up and the application version file is extracted.
commands:
  01_redis_install:
    # run this command from /tmp directory
    cwd: /tmp
    # don't run the command if phpredis is already installed (file /etc/php.d/redis.ini exists)
    test: '[ ! -f /etc/php.d/redis.ini ] && echo "phpredis extension not installed"'
    # executed only if test command succeeds
    command: |
      wget https://github.com/phpredis/phpredis/archive/4.0.2.zip -O phpredis.zip \
      && unzip -o phpredis.zip \
      && cd phpredis-* \
      && phpize \
      && ./configure \
      && make \
      && make install \
      && echo extension=redis.so > /etc/php.d/redis.ini \
      && service httpd restart

The

`commands`

line tells the EB platform the following lines are command we want to run on the initial of the platform. Detailed documents can be found at here. What the command sequences do is downloading a phpredis archive from the official release, then unzip the archive and install the extension. After installation, the command add the extension config in the file  /etc/php.d/redis.ini and reboot the httpd server.

Then on you working machine, you can zip both .ebextensions/ and html/ into one archive to deploy. One thing to notice is that if you work on Mac, the default archive process will include unnecessary files and directories such as “__MACOSX/.ebextensions/._php-modules.config”. These files can cause problems with the EB platform. To avoid these files, you can use command line to zip the directories. Open terminal and cd to the directory that contains html/. Then run:

zip -r -X archive.zip html .ebextensions

This will create an archive.zip next to html includes both html and .ebextensions.

Then go to EB and deploy the archive. If you have the file phpinfo.php in html/, you can access it to see if the redis extension is enabled with PHP.

Allow Connections from Outside to AWS Redis

The ElastiCache servers have two levels check for connections:
1. The connections are only allowed from within the same Subnet Group.
2. The security group redis cluster uses should allow access from port 6379 (the default port redis usually uses).

To do so, you will need to make sure the EB and ElastiCache are in the same subnet group. Then find out the security group ID (e.g sg-6xxxxxxx) the ElastiCache uses. Then go to EC2 section in AWS console. Find the Security Groups tag on the left menu. Then in the list of security groups, find out the group ID. Then edit the Inbound part of the group. Add one more item with follow parameters:

Type = Custom TCP Rule
Protocol = TCP
Port Range = 6379
Source = (You can use the security group of EB)

After save the settings, you can now access the redis server on you EB servers.

Test connection

To be able to test the connection, you will need to be able to SSH to one of the EC2 instance in your EB environment. To do so, you will need to set a “EC2 key pair” to the EB env. In the Configuration part of your EB, choose security, under “Virtual machine permissions”, you will find the “EC2 key pair” to assign. After assigning the key, the EB will have to replace all existing instances due to the security policies. But you don’t have to panic, the EB will do all the job for you.

Then you can use the .pem file you saved earlier for on your machine to SSH to one of the EC2 instance in EB environment.

Let’s assume we have SSH in one of the server. To test connection without PHP, you will need to install redis-cli. You can follow instructions on this page to install redis-cli only.

Then run

redis-cli -h master.xxxxxx.xxxxxx.use1.cache.amazonaws.com ping

If you get a response as PONG, it means you can connect to redis server properly. Note, “master.xxxxxx.xxxxxx.use1.cache.amazonaws.com” is the main entry point of your ElastiCache.

But if you get error like

Error: Connection reset by peer

It means you enabled the “Encryption in-transit” (TLS/SSL) for your ElastiCache server. The redis-cli does not support TLS/SSL yet. You will need to find another way to wrap the connection.

Encryption in-transit

A suggested method would be using “stunnel”. The solution is in on this page. Run:

sudo yum install -y stunnel

Will install the stunnel. Then you will need to put following content in its config file such as “/etc/stunnel/redis.conf”:

fips = no 
setuid = root 
setgid = root 
pid = /var/run/stunnel.pid 
debug = 7 
options = NO_SSLv2 
[redis-cli] 
    client = yes 
    accept = 127.0.0.1:6379 
    connect = master.xxxxxx.xxxxxx.use1.cache.amazonaws.com:6379

Then you can start the stunnel by:

sudo stunnel /etc/stunnel/redis.conf

Now you can try

redis-cli -h localhost

because the stunnel config help you connect to your redis cluster in TLS and open up a local port for you to use. It a security bridge from your EC2 machine to your redis cluster.

Try type “ping”. If you still got error like

(error) NOAUTH Authentication required.

It means you had setup Redis-Auth on your redis cluster, you will need to find out the password you set and use

auth 5xxxxxxxxxxxxxxxxxxx

with in the redis-cli to get authentication passed.

Wrap up for All EB instances

But all the previous settings are done in one EC2 instance, how do you ask EB to apply the changes across the EC2 cluster? Again, we need to add .ebextensions. In your .ebextensions/php-modules.config, add the following lines:

packages:
    yum:
        stunnel: []

files:
    "/etc/stunnel/redis.conf" :
        mode: "000644"
        owner: root
        group: root
            content: |
            fips = no
            setuid = root
            setgid = root
            pid = /var/run/stunnel.pid
            debug = 7
            options = NO_SSLv2
            [redis-cli]
                client = yes
                accept = 127.0.0.1:6379
                connect = master.xxxxxx.xxxxxx.use1.cache.amazonaws.com:6379

And then under commands section, you will need to add two more to stop and start the stunnel process:

    02_kill_old_stunnel:
        test: 'pgrep -u root stunnel && echo "There are existing stunnel process"'
        command: sudo pkill stunnel

    03_start_stunnel:
        command: sudo stunnel /etc/stunnel/redis.conf

Save the file and deploy them. Then you can connect to your redis on any of your EB instance.

PHP Test Connection

When it comes to php, we can simply put the following content in one php file in any of your instance

<?php

$redis = new Redis();

$redis->connect('localhost', 6379);
$redis->auth('5xxxxxxxxxxxxxxxxxxx');
echo $redis->ping();

Such as ~/test.php, then run

php -f test.php

with in the same directory to check the connection in PHP.

For Drupal

Finally, we just need to add following settings in drupal settings.php to get redis module running:

$conf['redis_client_interface'] = 'PhpRedis';
$conf['redis_client_host'] = '127.0.0.1';
$conf['redis_client_password'] = '5xxxxxxxxxxxxxxxxxxx';
$conf['lock_inc'] = 'sites/all/modules/contrib/redis/redis.lock.inc';
$conf['path_inc'] = 'sites/all/modules/contrib/redis/redis.path.inc'; 
$conf['cache_backends'][] = 'sites/all/modules/contrib/redis/redis.autoload.inc';
$conf['cache_default_class'] = 'Redis_Cache';
$conf['cache_prefix'] = 'site1';

Hope this can help anyone struggling with Drupal 7, EB and redis.

Geff’s been in design for nearly 20 years, in digital for the last 10. Geff uses User-Centred Design principles to develop CX and UX that delivers for clients. He loves problem solving, leading design sprints and prototyping. Geff advocates strongly for user feedback and enjoys the challenge of creating solutions that positively impact user behaviour and interaction with technology.

Get in touch, We love to talk