This wiki describes setting up consistent snapshots for EBS. It works with RAID arrays and a single volume. It also can be automated to run on a cron job.
Step 1: EBS Consistent Snapshots Installation
Instructions for installing ec2-consistent-backup can be found here: http://alestic.com/2009/09/ec2-consistent-snapshot.
Set-up access to Alestic PPA
codename=$(lsb_release -cs)
echo "deb http://ppa.launchpad.net/alestic/ppa/ubuntu $codename main"|sudo tee /etc/apt/sources.list.d/alestic-ppa.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BE09C571
sudo apt-get update
Install ec2-consistent-snapshot package
sudo apt-get install -y ec2-consistent-snapshot
sudo PERL_MM_USE_DEFAULT=1 cpan Net::Amazon::EC2
Step 2: ec2-consistent-snapshot for Mercury w/ RAID or Single Volume
I found some helpful code at Backtype which I've re-written to handle Mercury's defaults and with the assumption you've followed the tutorial on EBS+RAID. The same command could also be used for a single volume, just use one volume id instead of 4 (or however many volumes you may be using)
Example: (Instance Store with xfs-filesystem mountpoint)
Note: that the following command is all on one line
ec2-consistent-snapshot --aws-access-key-id <YOUR ACCESS KEY HERE> --aws-secret-access-key <YOUR SECRET KEY HERE> --region <YOUR REGION HERE, EXAMPLE: "us-east-1"> --mysql-username <YOUR MYSQL USER HERE> --mysql-password <YOUR MYSQL PASS HERE> --xfs-filesystem /vol <YOUR VOLUME IDS HERE, EXAMPLE: "vol-57f8002z vol-57f8002f vol-57f8002h vol-57f8002g" or only one volume example "vol-57f8002z">
You'll need the volume IDs for each volume (you can find these listed in your AWS console). These go at the very end of the command following /vol.
Example: (EBS Boot without xfs-filesystem mountpoint)
ec2-consistent-snapshot \
--aws-access-key-id <YOUR ACCESS KEY HERE> \
--aws-secret-access-key <YOUR SECRET KEY HERE> \
--region us-east-1 \
--mysql-username <YOUR MYSQL USER HERE> \
--mysql-password <YOUR MYSQL PASS HERE> \
--description "Consistent Snapshot $(date +'%Y-%m-%d %H:%M:%S')" \
<YOUR VOLUME IDS HERE, EXAMPLE: "vol-57f8002z">
Step 3: Set up a Cron Job to Execute Consistent Snapshots
Now I'd recommend adding this to a cron job so it gets done on a regular basis. You could create a cron job with the following steps:
- Create a text file somewhere on your instance (example location, your user directory)
- Add the above command to this file and save it
- Change the file to be executable
- Add the script to your cron
touch <path/to/user/scriptname>
pico <path/to/user/scriptname>
You can check to ensure the command is working by running the script manually from the command line.
<path/to/user/scriptname>
#Example (from current directory) ./create-consistent-ebs-snapshot
chmod +x </path/to/user/scriptname>
crontab -e
0 0,12 * * * /path/to/user/scriptname
Cron will email your system account if there are any errors with the command. You can check to ensure the command is working by running the script manually from the command line.
<path/to/user/scriptname>
If it works correctly you will see the ids of the snapshot backups that the script created (and of course you can see these in your AWS console).
if you get this error (via the cron daemon emails):
Can't exec "xfs_freeze": No such file or directory at /usr/bin/ec2-consistent-snapshot: ERROR: xfs_freeze -f /vol: failed(-1)
The following steps may resolve it:
echo $PATH
save the output from this command and run "crontab -e"
now enter the data from the above command as the first line of your cron file like this:
PATH=<data from echo $PATH>
This will give cron the same path variables as you have which is why the ec2 snapshot command might have worked when run from the command line but was causing errors when run under cron.
Step 4: Automating Snapshot Deletion
This php script may come in handy if you script your backup snapshots. It will delete old snapshots for you.
Check out (download) the delete old snapshots script with svn
cd <path/to/user/scriptname>
svn checkout http://ec2-delete-old-snapshots.googlecode.com/svn/trunk/ ec2-delete-old-snapshots-read-only
Update Amazon Access Key and Secret Access Key in "ec2-delete-old-snapshots.php" with your personal Amazon codes
Note: make sure to replace the everything in the single quotes, including angle brackets.
sudo nano <path/to/user/ec2-delete-old-snapshots-read-only>ec2-delete-old-snapshots.php
define('AWS_ACCESS_KEY_ID', '<Your Access Key ID>');
define('AWS_SECRET_ACCESS_KEY', '<Your Secret Access Key>');
Move Amazon php library to usr/share/php
sudo mv <path/to/user/>ec2-delete-old-snapshots-read-only/Amazon /usr/share/php/
Update Amazon Access Key and Secret Access Key in "usr/share/php/Amazon/EC2/Samples/.config.inc.php" with your personal Amazon codes
Note: make sure to replace the everything in the single quotes, including angle brackets.
sudo nano usr/share/php/Amazon/EC2/Samples/.config.inc.php
define('AWS_ACCESS_KEY_ID', '<Your Access Key ID>');
define('AWS_SECRET_ACCESS_KEY', '<Your Secret Access Key>');
Install PHP-XSL
sudo apt-get install php5-xsl
sudo apache2ctl restart
Configure cron to run automatically
- Create a text file somewhere on your instance (example location, your user directory)
- Add this command to this file and save it
- Change the file to be executable
- Add the script to your cron
touch <path/to/user/scriptname>
pico <path/to/user/scriptname>
php ec2-delete-old-snapshots.php -v vol-id [-v vol-id ...] -o days
#example php ec2-delete-old-snapshots.php -v vol-aabbccdd -v vol-bbccddee -o 7
chmod +x </path/to/user/scriptname>
crontab -e
0 0,12 * * * /path/to/user/scriptname
You can check to ensure the command is working by running the script manually from the command line.
<path/to/user/scriptname>
#Example (from current directory) ./delete-consistent-ebs-snapshot