Setting up EBS for Mercury [Beta]

You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!

Posted by nullvariable on October 31, 2009 at 3:14am
Last updated by jlmeredith on Thu, 2010-11-18 10:32

Once you've followed the steps outlined for installing your site on Mercury you'll then want to take your site to the next level using Elastic Block Store. This is going to keep your MySQL database(s) persistent should your server instance crash. Also make it easy to replicate data for testing etc.

There's a good generic Amazon doc page on this here.

Step 1 - Collect the necessary data

Note the following:

instance ID
region (example: us-east-1). Also be sure to note the letter following it.
access key ID
secret access key.

Step 2 - Create (Provision) the Volume

You can do this with command line tools or Elastic Fox, I've included a screen shot of the current AWS Management Console. Select the size you'll need the drive to be, choose the availability zone that matches the region your instance is in, including the letter as noted in step 1. If this is a new volume you won't be using a snapshot (we'll cover this later)

Step 3 - Connect the EC2 Instance to the EBS Volume

Part 1: attach the EC2 Instance to the EBS. Click the "Attach Volume" button and find the instance previously noted to link this drive to the server. Make sure you note the device name (example: /dev/sda) assigned to the volume, you'll need it in a minute.

Part 2: login via SSH, and mount the EBS volume. You'll need a ubuntu utility for this step so issue the following command:

sudo apt-get install -y xfsprogs

Now create an XFS file system on the EBS volume:

sudo modprobe xfs
sudo mkfs.xfs [device name]

Device name should look like "/dev/sda"

Depending on if you are using a Pantheon instance or if you have built from a vanilla instance along with what kernel you built your instance with, you may receive a warning when executing the modprobe command above. This is due to something being precompiled in the kernel (more techincal information needed here) and can possibly be ignored.

Now mount the EBS volume:

(replace /dev/sdh below with your device name)

echo "/dev/sdh /vol xfs noatime 0 0" | sudo tee -a /etc/fstab
sudo mkdir -m 000 /vol
sudo mount /vol

Congrats! You now have a working EBS volume and anything you save here will persist beyond the specific machine instance.

Step 4 - Move the MySQL Install to the EBS Volume

Stop the MySQL server.

sudo /etc/init.d/mysql stop

Now move all the database files to the EBS volume.

sudo mkdir /vol/etc /vol/lib /vol/log

sudo mv /mnt/mysql     /vol/etc/
sudo mv /var/lib/mysql /vol/lib/
sudo mv /var/log/mysql /vol/log/

And point everything to the EBS volume:

sudo mkdir /mnt/mysql
sudo mkdir /var/lib/mysql
sudo mkdir /var/log/mysql

echo "/vol/etc/mysql /mnt/mysql     none bind" | sudo tee -a /etc/fstab
sudo mount /mnt/mysql

echo "/vol/lib/mysql /var/lib/mysql none bind" | sudo tee -a /etc/fstab
sudo mount /var/lib/mysql

echo "/vol/log/mysql /var/log/mysql none bind" | sudo tee -a /etc/fstab
sudo mount /var/log/mysql

Start up your MySQL server again.

sudo /etc/init.d/mysql start

Now your MySQL files are secure against failure of the EC2 instance. The following steps are optional but will allow you to create incremental backup snapshots of the EBS volume. These snapshots aren't just great backups, they can also be used to create another EBS volume (perfect for spooling up a test server with real content that's almost fresh).

Step 5 - Move the Apache Doc root to the EBS Volume

# make the new doc root for apache
sudo mkdir /vol/var
# move instance doc root to new doc root area
sudo mv /var/www /vol/var/www

# make the new doc root for apache
sudo ln -s /vol/var/www /var/www
# restart apache so new doc root is used 
sudo /etc/init.d/apache2 restart

All previous steps

Here is sample initialization script to be used after a new EBS is created (essentially all the script code above in one spot).

#!/bin/bash
# This is the initial setup to create and move the mySQL database over to the EBS 
sudo apt-get update && sudo apt-get upgrade -y
sudo apt-get install -y xfsprogs 
sudo modprobe xfs
sudo mkfs.xfs /dev/sdh

echo "/dev/sdh /vol xfs noatime 0 0" | sudo tee -a /etc/fstab
sudo mkdir -m 000 /vol
sudo mount /vol

sudo /etc/init.d/mysql stop

sudo mkdir /vol/etc /vol/lib /vol/log

sudo mv /mnt/mysql     /vol/etc/
sudo mv /var/lib/mysql /vol/lib/
sudo mv /var/log/mysql /vol/log/

sudo mkdir /mnt/mysql
sudo mkdir /var/lib/mysql
sudo mkdir /var/log/mysql

echo "/vol/etc/mysql /mnt/mysql     none bind" | sudo tee -a /etc/fstab
sudo mount /mnt/mysql

echo "/vol/lib/mysql /var/lib/mysql none bind" | sudo tee -a /etc/fstab
sudo mount /var/lib/mysql

echo "/vol/log/mysql /var/log/mysql none bind" | sudo tee -a /etc/fstab
sudo mount /var/log/mysql

sudo /etc/init.d/mysql start


###############


# make the new doc root for apache
sudo mkdir /vol/var
# move instance doc root to new doc root area
sudo mv /var/www /vol/var/www

# make the new doc root for apache
sudo ln -s /vol/var/www /var/www
# restart apache so new doc root is used 
sudo /etc/init.d/apache2 restart

Step 6 - Configure Snapshots

This script takes care of most of the heavy lifting. I'll try to add more detailed instructions on how you can use it soon.
EC2 Consistent Snapshot
[Coming soon]

Step 7 - Automated Snapshots

(See above script)
[Coming soon]

Basics adapted from this AWS tutorial.
See also my guide to setting up EBS and RAID.

Starting New Instance with existing EBS

The above script provides a start to using an EBS. But at some point you will terminate the instance and want to restart.

After you have terminated your instance, the EBS is still out there ready to be used again. Here are some steps to reuse the EBS.

Startup a new instance.
Attach the EBS volume to the instance.
Login via SSH and run following script.

#!/bin/bash
# volume must be attached via EC2 tools before running through this
# example: ec2-attach-volume -d /dev/sdh -i INSTANCE-ID VOLUME-ID
# where INSTANCE-ID and VOLUME-ID are replaced with your instance and volume ids.
sudo apt-get update && sudo apt-get upgrade -y
sudo -E apt-get install -y xfsprogs
sudo apt-get install emacs22

sudo /etc/init.d/mysql stop

echo "/dev/sdh /vol xfs noatime 0 0" | sudo tee -a /etc/fstab
sudo mkdir -m 000 /vol
sudo mount /vol

sudo rm -R /mnt/mysql
sudo rm -R /var/lib/mysql
sudo rm -R /var/log/mysql

sudo mkdir /mnt/mysql
sudo mkdir /var/lib/mysql
sudo mkdir /var/log/mysql

echo "/vol/etc/mysql /mnt/mysql     none bind" | sudo tee -a /etc/fstab
sudo mount /mnt/mysql

echo "/vol/lib/mysql /var/lib/mysql none bind" | sudo tee -a /etc/fstab
sudo mount /var/lib/mysql

echo "/vol/log/mysql /var/log/mysql none bind" | sudo tee -a /etc/fstab
sudo mount /var/log/mysql

# attach to  the EBS doc root for apache
# remove instance doc root (not needed)
sudo rm -R /var/www

# make the new doc root for apache
sudo ln -s /vol/var/www /var/www
# restart apache so new doc root is used 
sudo /etc/init.d/apache2 restart

sudo /etc/init.d/mysql start

Rebooting with an EBS

You might ask, what about rebooting an instance that is attached to an EBS that was configured with above script code? Well, this may depend on how the AMI was configured. If the AMI is not setup to know that an EBS is to be mounted then the reboot will most likely fail/lock up the instance.

For example, using the Project Mercury AMI, the EBS initialization script works great. But if you try to reboot (tried on 2010 05 15 ) the instance the reboot will fail (BUG: soft lockup detected on CPU#0! will show up in system log) and require terminating the instance.

The good news is that you would just start a new instance, connect the EBS, run resume script, and you'll be up and running in short time.

Still looking for better solution for reboot.

Attachment	Size
AWS screenshot.png	12.15 KB