Altering the init.sh to support EBS-boot - Fixed!!!

I received the following email today and i thought it was a question many folks would like to know the answer to:

Is there a way to alter your mercury init.sh so it will ask the directory to move mysql and other files too?

This will help all those folks struggling with the EBS set up.

My simple solution will be to start up the cannonical EBS for Lucid, (which is in ext3), attach another EBS volume formatted in XFS, install mercury in the EBS boot, move mysql datadir to the XFS formatted EBS, move most logfiles to /mnt and if necessary, start another EBS volume for innodb logs.

Updated 7/12/10 with solution!

I agree that a solution is needed. First, a little history as to why we treat AWS different than other VPSs. The root filesystem on a AWS server is 10GB with a 150GB filesystem attached to /mnt (which is a little unusual - on a non-AWS server this would be attached to a directory within /mnt). In order to save as much space on the root filesystem as possible, we store certain data on the /mnt filesystem (again, AWS servers only):

/var/lib/mysql gets synlinked to /mnt/mysql/lib
/var/log/mysql gets synlinked to /mnt/mysql/log
mysql temp dir gets moved to /mnt/mysql/tmp
tomcat temp dir gets moved to /mnt/tomcat6/tmp
/var/lib/varnish gets moved to /mnt/varnish/lib

This solves the space issue, but doesn't fix the potential data-loss problem - both / and /mnt on instance-backed servers are volatile - if the server crashes or is shut down, the data on these filesystems are gone. The filesystems themselves are gone. 2 common solutions to the problem are mounting a EBS volume and storing important data there or (a more recently available option) using a EBS-boot instance (also called EBS-backed instance).

We've looked at adding a variable to the /etc/mercury/server_tuneables that would define the path to store these directories instead (ie, you could replace /mnt with something else). However, there is an issue here: this works the first time because we know the default path and can rsync the old path to the new path before doing the symlink. This would be much more difficult if /mnt was stored in a variable that could be changed because there would be no record of the what the value was before (and hence, where to sync the files from)

Besides, with EBS-boot instances, I would think the desired default behavior would be to leave the default paths of mysql, varnish and tomcat alone since a person can make their EBS-boot root filesytem whatever size they wish. This would give users of EBS-boot servers the most flexibility by allowing someone to move, say, mysql storage to a separate (attached) EBS volume or even RDS.

To achieve this solution we need a way to reliably and automatically identify whether an AWS server is instance or EBS-backed.

Anyone know of a way or have another suggestion?

UPDATE 7/12/10 - we added a bcfg2 group to Mercury 1.1 Beta called mercury-aws-ebs (see http://groups.drupal.org/node/70268) which set's up all the aws-specific packages but makes no changes to the file structure - this should be a good solution for AWS. We've also added tuneables for services such as mysql allowing one to utilize Amazon's RDS without making major changes to Mercury.

Greg