===============================================================================
Updated... for Latest Mercury 1.1 release.
However it died after verifying the first Aegir site; I dont have the time to find out why at the moment so please use the previous revision which is known to work with Mercury 1.0 and the latest Aegir. I couldn't revert this wiki page to a previous date so this note will have to suffice.
===============================================================================
===============================================================================
These instructions are based on Ubuntu which the Mercury AMI uses, zero considerations are made for other os's
We can keep working on this until its good enough to base a new set of the bcfg2 mercury configs and also
to create a more generalized AMI thats easy to turn on.
For now its a bit manual but goes fast once you get used to it.
Main Aegir + Mercury Wiki page
• http://groups.drupal.org/node/55033
__________________________________________________________________________________________
__________________________________________________________________________________________
(1) Install AMI and the default Pressflow site
(2) Attach Drives
(2.1) Move MySQL to EBS
(3) Configure some system basics
(4) Aegir
(5) Consistent Snapshots for AWS
(6) Multi-core Solr (incomplete)
(7) Update/Migrate existing Aegir + Mercury to new Server
__________________________________________________________________________________________
__________________________________________________________________________________________
(1) Install AMI and the default Pressflow site
------------------------------------------------------------------------------------------
Use AWS management console to launch a new Mercury instance, I suggest a large for multisites, if your deving use a spot instance its cheaper.
After the AMI boots up wait a few min until the bcfg2 script performs its updates.
You will also have to setup your ssh access, an elastic IP so you can point your domains to the box and also the security group bit. Follow the getting started AWS EC2 instructions if your not familiar. The latest Mercury step-by-step wiki is at http://groups.drupal.org/node/70268
Even though I wont be using the pressflow site included in the AMI I still install it to check that the system is working
and see whatever changes have been made to the various settings. Especially the settings.php file.
Point your browser to the ami & Run through the installation process.
__________________________________________________________________________________________
__________________________________________________________________________________________
(2) Attach Drives
------------------------------------------------------------------------------------------
Create two drives in the in management console make note of the vol-ids
then attach the two drives to the ami, taking note of the /dev/sd* you attach with. Normally, the AMI has two mounts, /dev/sda1 and /dev/sda2, for the root drive and the larger /mnt drive, leaving the user a choice from /dev/sdc to /dev/sdp.
[Add instructions for formatting drives here]
For EBS, the XFS file system is preferred due to its compatibility with ec2-consistent-snapshot. The instructions below are for formatting the EBS drives in XFS
mkfs.xfs /dev/sd* - do this twice substituting your actual locations noted when creating and attaching the drives aboveNow add the drives to your fstab
//webroot drive
echo "/dev/sdo /vol xfs noatime 0 0" | sudo tee -a /etc/fstab
sudo mkdir -m 000 /vol
sudo mount /vol//db drive
echo "/dev/sdp /db xfs noatime 0 0" | sudo tee -a /etc/fstab
sudo mkdir -m 000 /db
sudo mount /db__________________________________________________________________________________________
__________________________________________________________________________________________
(2.1) Move MySQL to EBS
------------------------------------------------------------------------------------------
//stop mysql
/etc/init.d/mysql stop
//move the data to the db drive
mkdir /db/etc /db/lib /db/log
mv /mnt/mysql /db/etc/
mv /var/lib/mysql /db/lib/
mv /var/log/mysql /db/log/sudo mkdir /mnt/mysql
sudo mkdir /var/lib/mysql
sudo mkdir /var/log/mysql
echo "/db/etc/mysql /mnt/mysql none bind" | tee -a /etc/fstab
mount /mnt/mysql
echo "/db/lib/mysql /var/lib/mysql none bind" | tee -a /etc/fstab
mount /var/lib/mysql
echo "/db/log/mysql /var/log/mysql none bind" | tee -a /etc/fstab
mount /var/log/mysql//Start up your MySQL server again.
/etc/init.d/mysql start__________________________________________________________________________________________
__________________________________________________________________________________________
(3) Configure some system basics
------------------------------------------------------------------------------------------
//set the mysql root password
mysql -u root
set password for root@localhost=PASSWORD('yournewpass');
flush privileges;
\q//PhpMyAdmin
https://help.ubuntu.com/community/phpMyAdmin
apt-get install phpmyadmin
add the following to /etc/apache2/apache2.conf
# phpmyadmin
Include /etc/phpmyadmin/apache.conf//then reload apache
/etc/init.d/apache2 reload__________________________________________________________________________________________
__________________________________________________________________________________________
(4) Aegir
------------------------------------------------------------------------------------------
New alpha8 version assumed in below instructions, they were taken from the install.txt found here
http://groups.drupal.org/aegir-hosting-system/documentation
//Install Needed Additions as per the Aegir install instructions NOTE: some of these are in already which shouldn't matter just run it.
apt-get install apache2 php5 php5-cli php5-mysql mysql-server postfix
apt-get install sudo git-core unzip//remove the installed drush in the pressflow site, it will get in the way.
rm -r /var/www/pressflow/sites/all/modules/drush
//add Aegir User & make aegir a user of group www-data, be sure the --home dir matches the new webroot drive you made
adduser --system --group --home /vol/aegir aegir
adduser aegir www-data//Add Aegir user to sudors
visudoAdd the following at the bottom
aegir ALL=NOPASSWD: /usr/sbin/apache2ctl//Apache
a2enmod rewrite
echo "Include /vol/aegir/config/apache.conf" > /etc/apache2/conf.d/aegirMySQL
You want to run these commands using your mysql 'root' user, substituting the 'XXXXXXXX' with a real password.
login to mysql
mysql -u root -p
once your in do the following
create aegir db and db user
CREATE DATABASE aegir;
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, \
CREATE TEMPORARY TABLES, LOCK TABLES ON aegir.* TO \
'aegir'@'localhost' IDENTIFIED BY 'XXXXXXXX';// Now Create Super Aegir user
GRANT ALL PRIVILEGES ON *.* TO 'aegir_root'@'localhost' \
IDENTIFIED BY 'XXXXXXXX' WITH GRANT OPTION;
exit//Restart mysql & apache
/etc/init.d/mysql restart//Install the Aegir files
Open and copy the install script and save an edited copy to the /vol/aegir directory as install.sh.tx
install.sh for alpha8 can be found here --> http://groups.drupal.org/aegir-hosting-system/documentation
//You need to change the domain to your domain
AEGIR_DOMAIN=aegir.example.com
//now run the install script
su -s /bin/bash aegir -c "sh install.sh.txt"//when its done restart apache then go to the link posted in the output of the successful install.sh.txt run
//but first restart apache
/etc/init.d/apache2 restartTake out any reference to port 80 and replace with 81 or it wont work. Specifically in the /vol/aegir/config/vhost.d/aegir.ursite.com vhost file for your main aegir site, also in the /vol/aegir/config/apache.conf file. You MUST do this before going to the remaining step below .. otherwise your pissing in the dark since ports wont match up and well nothing will work the way it should. lol
Ok... You will need to follow the remaining steps in the Aegir install wizard as usual so.. Point your browser to http://$AEGIR_DOMAIN/install.php and proceed with the remainder of the installation using the Hostmaster Install profile. Some of the instructions given, you will already have completed, but carefully read each step in turn to ensure you didn't miss anything.
Create Pressflow Platform in Aegir
Assuming all went well you will now need to create a pressflow platform within drupal.
cd /vol/aegir
git clone git://github.com/damienmckenna/pressflow-6.git
#This repository seems to no longer exist
#git clone git://github.com/bigmack83/pressflow-6.gitMove any modules/themes etc you want in the platform to
/vol/aegir/pressflow-6/sites/all/modules
/vol/aegir/pressflow-6/sites/all/themes
you can use drush to dl them if you want or ftp.. whatever you like
Now go to your Aegir site and add the platform
yoursite.net/node/add/platform
make sure the publish path is the full path.
/vol/aegir/pressflow-6
Save it and wait for the verify to complete then add sites using the new pressflow platform.
Custom settings.php with Aegir
In order to make memcached work in aegir (direct drupal cache's to memcached) you need to customize your settings.php which is not out of the box possible withe Aegir; the reason being is that every time you run a Verify task on a site or platform the settings.php files are all regenerated and you loose your customizations. ... there is a workaround.
After installing Aegir you will need to...
//disable Aegir's default forced cache settings
file is found here--> /vol/aegir/.drush/provision/platform/provision_drupal_settings.tpl.php
comment out line this line like such-- #$conf['cache'] = 1;
Create a global.inc file in aegir's main includes dir /vol/aegir/config/includes/global.inc
nano /vol/aegir/config/includes/global.incThen copy over your bits from settings.php in the main pressflow site included in the AMI, these should go into a preg_match statement.
I also set the offline theme, anything additional you'd place in settings.php should go here for each site that needs it.
Here is an example with two sites. NOTE: You wont need this for the actual aegir site, only for the sites you want to use external cache, or to make changes to their settings.php
<?php //global settings.php
$thishost = $_SERVER[HTTP_HOST];
//Your site
if(preg_match("/(\yoursite\.com)/", $thishost)) {
$conf['maintenance_theme'] = 'ur_theme_choice';
# Simpletest goodies from pressflow
$GLOBALS['simpletest_installed'] = TRUE;
if (preg_match("/^simpletest\d+$/", $_SERVER['HTTP_USER_AGENT'])) {
$db_prefix = $_SERVER['HTTP_USER_AGENT'];
}
# Varnish reverse proxy on localhost
$conf['reverse_proxy'] = TRUE;
$conf['reverse_proxy_addresses'] = array('127.0.0.1');
# Memcached configuration - uncomment to use Memcached instead of Cacherouter:
$conf = array(
'memcache_key_prefix' => 'yoursite',
'cache_inc' => './sites/all/modules/aa_pressflow/memcache/memcache.db.inc',
'memcache_servers' => array(
'127.0.0.1:11211' => 'default',
'127.0.0.1:11212' => 'menu',
'127.0.0.1:11213' => 'filter',
'127.0.0.1:11214' => 'form',
'127.0.0.1:11215' => 'block',
'127.0.0.1:11216' => 'update',
'127.0.0.1:11217' => 'views',
'127.0.0.1:11218' => 'content',
'127.0.0.1:11219' => 'apachesolr',
),
'memcache_bins' => array(
'cache' => 'default',
'cache_menu' => 'menu',
'cache_filter' => 'filter',
'cache_form' => 'form',
'cache_block' => 'block',
'cache_update' => 'update',
'cache_views' => 'views',
'cache_views_data' => 'views',
'cache_content' => 'content',
'cache_apachesolr' => 'apachesolr',
),
);
}
//Another site
if(preg_match("/(dev\.somesite\.net)/", $thishost)) {
$conf['maintenance_theme'] = 'ur_theme_choice';
# Simpletest goodies from pressflow
$GLOBALS['simpletest_installed'] = TRUE;
if (preg_match("/^simpletest\d+$/", $_SERVER['HTTP_USER_AGENT'])) {
$db_prefix = $_SERVER['HTTP_USER_AGENT'];
}
# Varnish reverse proxy on localhost
$conf['reverse_proxy'] = TRUE;
$conf['reverse_proxy_addresses'] = array('127.0.0.1');
$conf = array(
'memcache_key_prefix' => 'anothersite',
'cache_inc' => './sites/all/modules/aa_pressflow/memcache/memcache.db.inc',
'memcache_servers' => array(
'127.0.0.1:11211' => 'default',
'127.0.0.1:11212' => 'menu',
'127.0.0.1:11213' => 'filter',
'127.0.0.1:11214' => 'form',
'127.0.0.1:11215' => 'block',
'127.0.0.1:11216' => 'update',
'127.0.0.1:11217' => 'views',
'127.0.0.1:11218' => 'content',
'127.0.0.1:11219' => 'apachesolr',
),
'memcache_bins' => array(
'cache' => 'default',
'cache_menu' => 'menu',
'cache_filter' => 'filter',
'cache_form' => 'form',
'cache_block' => 'block',
'cache_update' => 'update',
'cache_views' => 'views',
'cache_views_data' => 'views',
'cache_content' => 'content',
'cache_apachesolr' => 'apachesolr',
),
);
}__________________________________________________________________________________________
__________________________________________________________________________________________
(5) Consistent Snapshots for AWS
------------------------------------------------------------------------------------------
Install ec2-consistent-snapshot
http://alestic.com/2009/09/ec2-consistent-snapshot
//setup PPA - Ubuntu
codename=$(lsb_release -cs)
echo "deb http://ppa.launchpad.net/alestic/ppa/ubuntu $codename main"|
sudo tee /etc/apt/sources.list.d/alestic-ppa.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BE09C571
sudo apt-get update//Now install ec2-consistent-snapshot
sudo apt-get install -y ec2-consistent-snapshot//Still need the perl thing
sudo PERL_MM_USE_DEFAULT=1 cpan Net::Amazon::EC2//see the doc
man ec2-consistent-snapshotHere are some good instructions by nullvariable that I got this all from.
http://groups.drupal.org/node/36766
Cron task - Make sure the region and the mySQL user etc match whats needed too.
MAKE SURE YOU CHANGE THE vol-ids to match your drives drives at the end of these commands, and they also must be only one line each.
//drive with mysql
ec2-consistent-snapshot --aws-access-key-id XXXXXXXXXXXXXXXXX --aws-secret-access-key XXXXXXXXXXXXXXXXX --region us-east-1 --mysql --mysql-username root --mysql-password XXXXXXX --xfs-filesystem /db vol-2334d4a//basic drive such as the webroot
ec2-consistent-snapshot --aws-access-key-id XXXXXXXXXXXXXXXXX --aws-secret-access-key XXXXXXXXXXXXXXXXX --region us-east-1 --xfs-filesystem /vol vol-27gt6d48//You could create a cron job with the following steps:
1. create a text file somewhere on your instance (example location, your user directory) and place the above code in it
touch /root/snapshots2. add the above ec2-consistent-snapshot commands to this file and save it
pico /root/snapshots3. change the file to be executable
chmod +x /root/snapshots4. add the script to your cron
crontab -e//add this for snaps twice a day
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:
MAILTO=you@uremail.com
0 0,12 * * * /root/snapshotsCron will email your system account every time it performs. You can check to ensure the command is working by running the script manually from the command line. If it works correctly you will see the ids of the snapshot backups that the script created (and of course you can see them in your AWS console too). Its nice to get the emails for these because if they ever stop being created you open a window of pain if the data gets corrupted. Snapshots are very useful; at any time you can create new drives from them from within the aws management console & spin up new servers.
Auto-delete old snapshots? ---haven't used this yet personally
//This php script may come in handy if you script your backup snapshots:
http://code.google.com/p/ec2-delete-old-snapshots/
__________________________________________________________________________________________
__________________________________________________________________________________________
(6) Multi-core Solr
------------------------------------------------------------------------------------------
http://groups.drupal.org/node/72488
__________________________________________________________________________________________
__________________________________________________________________________________________
(7) Update/Migrate existing Aegir + Mercury to new Server
------------------------------------------------------------------------------------------
will post link to that wiki page here.
__________________________________________________________________________________________