Step-by-step: Setting up Varnish, Apache, APC and Solr Project Mercury Style

By popular request, here are step-by-step instructions for building the Project Mercury setup on a fresh ec2 image. We have both 32 and 64 bit versions of this AMI available (see http://groups.drupal.org/amazon-web-services-s3-ec2).

This is a wiki page which I will try to keep up to date as the project evolves. Please feel free to comment, add notes, and correct any mistakes you see.

Updated on 9/20/09 with numerous fixes and the new pressflow BZR location!
Updated on 10/8/09 to include adding Apache Solr to the install
updated on 10/22/09 with configuration file download instructions
Updated on 11/14/09 with restructured instructions
Updated on 11/19/09 for Mercury 0.8-Beta

Project Mercury Step By Step

1) Basic startup

a) Launch a new Ubuntu image. I use alestic ubuntu-9.04-jaunty-base.
b) Add the bzr/bzrtools ppa as per https://launchpad.net/~bzr/+archive/ppa
c) Get updates of installed packages and install needed packages:

Note: For our Mercury AMIs we leave the mysql root password blank (so the owner can choose it), but you should pick one during the install. Also we include postfix in the above instructions – replace this if you have a preference for another Mail Transport Agent (e.g. exim, sendmail).

2) Optional: Download config files:

If you choose, we have copies of all the config files that you'll need to edit available on launchpad.net – already edited!. Once downloaded, simply copy the files to the appropriate path instead of modifying the original. Make backups of all original config files before modifying and/or copying over them. Also note that copying the config files out of order (or all in one step rather than at the appropriate time in the instructions) will cause errors. You can skip this step if you wish to edit the config files by hand. We've only uploaded config files of Ubuntu Jaunty so far - we'll be adding additional distributions soon

3) Configure apache and varnish to work together:

a) Change port 80 to port 8080 in /etc/apache2/ports.conf
b) Change port 80 to port 8080 in /etc/apache2/sites-available/default
c) Make a pressflow directory for varnish ( mkdir -p /var/lib/varnish/pressflow; chown varnish.varnish /var/lib/varnish/pressflow)
d) Edit /etc/default/varnish and:

e) To allow Drupal's .htaccess file to function, edit /etc/apache2/sites-available/default and change "AllowOverride None" to "AllowOverride All" in the Directory /var/www/ block.
f) Enable mod-rewrite, restart apache and restart varnish (a2enmod rewrite; /etc/init.d/apache2 restart; /etc/init.d/varnish restart)
g) Load the public DNS of your instance. You should get the basic "It works!" page. This is coming via Varnish on port 80, pulled from Apache on the backend!
h) (Extra credit) install the Live HTTP headers firefox plugin and look for headers like

Then try loading it with the port 8080 attached (e.g. http://public_AWS_dns:8080). Note the differences.

To do step 3 using the config files we provided:

4) Set up APC:

a) Edit /etc/php5/conf.d/apc.ini and make sure it has the following:

b) If you're setting up a stable (rather than a development) server, consider adding the apc.apc.stat = 0 option as well. This will prevent APC from re-checking cached files, and improves performance. However, if you're changing files frequently (e.g. the site is still under development), it's not a good idea. For more information, see this 2bits article.

c) Increase the max memory available to php by changing memory_limit = 16M to memory_limit = 64M in /etc/php5/apache2/php.ini.

To do step 4 using the config files we provided:

5) Configure Mysql:

a) Create a pressflow database
b) Grant access to a non-root user
c) Optional: Change the default storage engine from MyISAM to InnoDB and restart apache (add default-storage-engine=innodb to /etv/my/my.conf)

6) Download and configure Pressflow:

We're going to replace the "It works!" /var/www directory with a BZR checkout of pressflow (from its new home on lauchpad):

a) Delete the /var/www directory (rm -r /var/www)
b) Download pressflow into /var/www (bzr branch lp:pressflow /var/www)
c) Make files dir for Pressflow and set ownership and permissions (mkdir /var/www/sites/default/files; chown -R root:www-data /var/www/sites/default/; chmod -R 775 /var/www/sites/default/)

To recap step 6:

7) Configure Varnish to work with Pressflow:

Edit /etc/varnish/default.vcl and add this, replacing the included "backend default" declaration:

Then /etc/init.d/varnish restart

This will work to basically configure Varnish and Drupal to work together in terms of serving cached pages.

To do step 7 using the config files we provided:

8) Install and configure Cacherouter:

a) Download Chacherouter from http://drupal.org/project/cacherouter, untar and move to Pressflow's module directory
b) Add the following to the bottom of /var/www/sites/default/settings.php

While we are at it, let's set the reverse-proxy settings:

To do step 8 using the config file we provided:

9) Install and configure Apache Solr

a) Install Tomcat6, the JAVA servlet container for Solr (apt-get install tomcat6)
b) Change #TOMCAT6_SECURITY=yes to TOMCAT6_SECURITY=no in /etc/default/tomcat6 (note that we also uncommented this line)
c) Download and install Apache Solr. We recommend using the nightly builds as Solr development is still pretty rapid

d) Insert the following text into /etc/tomcat6/Catalina/localhost/solr.xml (this tells Tomcat where to find Solr):

e) Replace all instances of 8080 with 8180 in /etc/tomcat6/server.xml (since 8080 is being used by apache)
f) Download the Apache Solr module from http://drupal.org/project/apachesolr, untar and move to Pressflow's module directory

g) Download php files for the Apache Solr module:

h) Set permissions for Apache Solr dir (chown -R tomcat6:root /var/solr/)
i) Restart tomcat (/etc/init.d/tomcat6 restart)

To do step 9 using the config files we provided:

10) Optional: Download ec2-metadata and configure Postfix:

a) We recommend installing the very handy ec2-metadata shellscript. It makes finding out info about your instance a breeze.

b) If you insalled Postfix, configure using the following:

c) Restart Postfix (/etc/init.d/postfix restart)

To recap step 10:

11) Install Pressflow:

a) Connect to your AMI using a web browser and install Pressflow

b) Go to Administer > Modules and enable Apache Solr framework, Apache Solr search, Search and CacheRouter via web interface

c) Go to Site configuration > Apache Solr and set:
Solr Port: 8180
Number of items to index per cron run: 50
Make Apache Solr Search the default: Enabled
Enable spellchecker and suggestions: checked

d) Go to Site configuration > Search Settings and set:
Number of items to index per cron run: 50

e) Go to Site configuration > Performance and set:
Caching mode: Aggresive
Minimum cache lifetime: 10 minutes
Page cache maximum age: 1 hour
Page compression: Enabled
Block cache: Enabled
Optimize CSS files: Enabled
Optimize JavaScript files: Enabled

f) Go to User Management -> Permissions and set:
search content: anonymous user and authenticated user
use advanced search: anonymous user and authenticated user

g) Change the permission of settings.php to protect it (chmod 755 /var/www/sites/default)

12) Setup the /mnt directory for storage, enable cron and create a boot script

Since root storage on AMIs are limited, we recommend using the /mnt directory for storage of Mysql files and the Varnish cache. We will soon have optional instructions for using Amazon's Elastic Block Storage (EBS) for storage.

a) Change the Mysql datadir from /var/log/mysql to /mnt/mysql in /etc/mysql/my.cnf.

b) Change the Mysql tmpdir from /tmp /mnt/tmp in the same file.

c) Change /var/lib/varnish/$INSTANCE/varnish_storage.bin to /mnt/varnish/$INSTANCE/varnish_storage.bin in /etc/default/varnish

d) Make the temp dir for Mysql (mkdir /mnt/tmp && chmod 666 /mnt/tmp)

e) Create a file (/erc/cron.d/drupal) with the following lines (this automates updating the Apache Solr database and other Drupal maintenance:

f) Set up a one-time boot script by adding the following to /etc/rc.local (keeping "exit 0" at the end of the file):

g) Create the directory for mercury (mkdir /etc/mercury), add the following to /etc/mercury/init.sh and make it executable (chmod +x /etc/mercury/init.sh):

To do step 12 using the config files we provided (only do one of the my.cnf commands!):

13) Repackage Your AMI

Here's my basic AMI packaging script based on Eric Hammond's work. In order to make this run, you need to put your certs in /tmp. See Eric's great blog post for more details: