Step-by-step: Installing Project Mercury 1.1 (Beta) on Ubuntu Lucid

Here are step-by-step instructions for building Project Mercury 1.1 (BETA!!!) on a fresh Ubuntu Lucid server using the configuration manager BCFG2. Project Mercury is known to work on AWS, Rackspace, RackspaceCloud and Linode. We have AMIs of Mercury 1.1 Beta for AWS EC2 (see: http://groups.drupal.org/node/71943) and (soon) a Linode script for Mercury 1.1 (both on Ubuntu Lucid).

6/1/10: Updates for multi-site support
7/12/10: Added EBS-boot and hudson support

Project Mercury Step By Step (using the BCFG2 configuration manager)

1) Start Ubuntu Lucid (10.04):

Our work is currently based on Lucid distribution of Ubuntu. If you want to use Amazon for your infrastructure there are many AMIs available, we prefer the ones provided by Canonical. View a list of AMIs at http://alestic.com/. We currently have Mercury 1.1 Beta config files for Ubuntu Lucid and CentOS 5.5 running on AWS, Rackspace and Linode. Make sure that your slice, linode, virtual box (et al) is at least 12gb.

2) Configure apt sources and get updates:

Here we download some apt sources lists to make sure everyone is on the same page:

Note: 2014 11 16 this URL is not longer valid http://pantheon-storage.chapterthree.com/mercury.list.1.1

It seems apt-get can't find Hudson...Found this out in Step 10

Replace 'deb http://pkg.hudson-labs.org/debian binary/' with 'deb http://hudson-ci.org/debian binary/' <== I had to use http://hudson-ci.org/downloads/debian/
reference: http://wiki.hudson-ci.org/display/HUDSON/Installing+Hudson+on+Ubuntu

Replace 'archive.ubuntu.com' and 'security.ubuntu.com' with 'old-releases.ubuntu.com' on all lines referencing a 'karmic' repository
reference: http://pim.famnit.upr.si/blog/index.php?/archives/225-Update-an-old-Ubun...

You may see "W: Duplicate sources.list entry" errors during the apt-get update - these are innocuous and occur because we do have some duplication in /etc/apt/sources.list.d/mercury.list (vs. /etc/apt/sources.list) to make sure everyone is starting from the same point regardless of VPS and the default set of apt-sources decided upon by that VPS. You may also be asked to configure grub-pc (AWS only) - leave the line blank. You will then be asked if it's ok to "Continue without installing GRUB" - say "Yes".

If you prefer aptitude to apt, you'll want to copy the contents of /etc/apt/preferences.d/lucid into /etc/apt/preferences due to a bug in aptitude (see http://groups.drupal.org/node/75003).

3) Install and configure BCFG2:

Time to install BCFG2 and BZR and some other packages used by Mercury and configure BCFG2:

**note - If you are installing the latest (as of 8 January 2011) Ubuntu Lucid 64 bit image in Amazon AWS, or locally using vmware, you may also run into the following error:

Please follow the thread at http://groups.drupal.org/node/113989#comment-379919 for a work around and then put in the command above (again).

If you do not get the error or after using the workaround, proceed as follows:

Will result in the following questions (recommended answers in bold) where (accept default) means hit return/enter):

Store bcfg2 configuration in [/etc/bcfg2.conf]: (accept default)
Location of bcfg2 repository [/var/lib/bcfg2]: (accept default)
Directory /var/lib/bcfg2 exists. Overwrite? [y/N]: y
Input password used for communication verification (without echoing; leave blank for a random):(accept default)
What is the server's hostname [hostname]: localhost
Input the server location [https://hostname:6789]: https://localhost:6789
Input base Operating System for clients:
1: Redhat/Fedora/RHEL/RHAS/Centos
2: SUSE/SLES
3: Mandrake
4: Debian
5: Ubuntu
6: Gentoo
7: FreeBSD
: 5

You may be prompted to input your
Country (Two Letter Code)
State (Full Name)
Locality Name (eg, city)

Warning: /etc/bcfg2.conf already exists. Overwrite? [y/N]: y
Generating a 1024 bit RSA private key
............................................................++++++
.................................++++++
writing new private key to '/etc/bcfg2.key'
-----
Signature ok
subject=/C=US/ST=Illinois/L=Argonne/CN=localhost
Getting Private key
Repository created successfuly in /var/lib/bcfg2

If you receive a message, "Error global name 'keypath' is not defined occured while trying to write configuration file to '/etc/bcfg2.conf'.", see the discussion at http://groups.drupal.org/node/145919 for a possible work-around.

Now we want to download the BCFG2 config files from launchpad.
If you see "You have not informed bzr of your Launchpad ID..." after the bzr command, launchpad is simply telling you that you need to login to "upload" - which we're not doing here - so you can safely ignore this message.

Ensure that bcfg2-server isn't running

If bcfg2-server is running, try stopping the server and check again to see if it's running:

If it is, kill all the bcfg2 processes:

4) Start the BCFG2 server:

Here we start the bcfg2 server and wait until it spins up (this will require several minutes) - you know the server is ready when you see "serve_forever() [start]" in the syslog (the tail -f command monitors the file).

When the output stops, you can control-c to exit the tail and move on to the next steps.

5) Start the BCFG2 client:

Note that bcfg2 -vqed can take some time (can take 15 minutes or more!) - it's done when the command prompt returns:

Non-AWS servers:

*if you replaced the location of the Hudson package in Step 2, then you may get a warning:

Don't worry, it will continue...

AWS* servers:

EBS-boot AWS* servers:

• *The -p 'mercury-aws[-ebs]' parameter is required for AWS servers only during installation (now), and is not required on subsequent starts of the bcfg2 client--`"sudo bcfg2 -vqed' will work just fine.

If you get the error "Server failure: Protocol Error: 401 Unauthorized" make sure the first line in your /etc/hosts file looks like this:
Ensure that localhost is before localhost.localdomain!

6) Install Drush:

Drush is a very useful command-line Drupal management system (see http://drupal.org/project/drush). The latest version of Drush includes Drush Make:

7) Install Mercury:

Here we ask drush to download code from our project page on launchpad.net and install some needed Drupal modules.

if you recieve an error, use the following:

8) Install ApacheSolr:

We replace Drupal's built-in search with ApacheSolr which is faster, less resource-intensive and modular:

9) Prepare Pressflow files and dirs:

Pressflow (http://pressflow.org/) is a distribution of Drupal with integrated performance, scalability, availability, and testing enhancements:

Proceed as follows:

10) Add hudson to sudoers and restart hudson:

Initially, we're only using hudson for running init.sh, cron.php and bcfg2 -vqed but there's much more coming...

11) Run the Mercury Init Hudson job

Hudson provides an interface to some system scripts including the Mercury Init script. To run the script log into hudson by directing your browser to http://yourdomain.tld:8081 and click on "Log In" in the upper right hand corner. Hudson is setup to use the system login/password. Some virtual servers providers, like AWS, don't have a password. You can change this by ssh'ing to the server and typing:

Here, you can also follow the instructions at https://getpantheon.com/support/accessing-and-using-hudson-pantheon-server and open an ssh tunnel to Hudson. However the instructions at Getpantheon.com provides a different port to Hudson. Specifically for Amazon AWS users you will need to change it as follows:

NOTE: For those not familiar with ssh tunneling, the above command is being done on your local computer (laptop, desktop, etc), not the server on which you are setting up Mercury. So locally you are setting up an ssh tunnel to the server being setup. The ssh tunnel command enables the link http://localhost:8081/ to communicate with Hudson on the Mercury server.

Once logged in, click on "Mercury Init" and then "Build Now" which does the following:

1. setup mysql, tomcat and varnish to use /mnt to save space on / (which is only 10GB on a EC2 small instance) - AWS servers only.
2. configure postfix with a usable hostname and check for (and download) updates to mercury and pressflow (and apply the mercury updates via BCFG2).
3. configure APC, varnish, tomcat and PHP memory usage based on the system memory using the /etc/mercury/config_mem.sh script
4. send a unique but non-identifying hash to us so we can get an idea of how many Mercury users there are.

You can follow the progress of the script by clicking on the running job (lower left corner) and then "Console Output".

If the Hudson script fails, read this issue.

If you are on a VM with a burstable RAM solution (eg. Virtuozzo), Mercury can get an inaccurate reading of your available RAM due to the way the VM handles the burstable RAM resources. If running "free -m" reports a total Mem far greater than what you'd expect, it's possible it's including the burstable RAM in it's calculation. In this case, Mercury will not be able to automatically configure the APC, Varnish, etc memory limits appropriately. To correct, manually update your /etc/mercury/server_tuneables to appropriate values based on your server's guaranteed RAM, and allow bcfg2 to update your configuration with the following command:

12) Configure Pressflow:

1. Go to your site in your web browser
2. Choose the Mercury profile (This will setup the appropriate modules and settings.)
3. Database name = "pantheon"
4. Database username ="root" (no password yet - we set it below)

After pressflow is configured:

5. Set the mysql root password and create a non-root account, changing new_user and new_password to appropriate values):
# Login to MySQL as root
mysql -u root
# Set root user password
mysql> set password for root@localhost=PASSWORD('new_password');
# Create new user and give a password
mysql> create user 'new_user'@'localhost' identified by 'new_password';
# Set privileges for new user
mysql> grant all on pantheon.* to 'new_user'@'localhost';
mysql> flush privileges;
mysql> \q
6. Update your Pressflow install with the new mysql account information by editing
   /var/www/sites/default/settings.php (again using the appropriate new_user and new_password values):
sudo nano -w /var/www/sites/default/settings.php

And change:

$db_url = 'mysqli://username:password@localhost/databasename';

to what you set up for the new user above. Make sure to edit the 'mysqli:' and not the 'mysql:' portion.

$db_url = 'mysqli://new_user:new_password@localhost/pantheon';

Next Steps

• If you're interested in using the new multi-site ability, see http://groups.drupal.org/node/72488
• To see what can be configured on your Mercury server, see http://groups.drupal.org/node/70258
• We using hudson to run drupal/drush cron jobs. We intend to add much more to hudson in the future. If you choose to turn it off via the tuneable, then run the following to make sure cronjob gets run:

  
sudo crontab /etc/mercury/templates/cron


• Time to populate your new Mercury server!