Posted by R-H on June 19, 2010 at 11:42am
I've followed the Step by Step Mecury 1.1 Wiki (http://groups.drupal.org/node/70268) successfully and would now like to create an AMI of this instance. It's root device type is EBS and I began with the Alestic AMI (Ubuntu 10.04 Lucid Canonical, ubuntu EBS boot: ami-714ba518). I used the GUI at the AWS management console and followed their instructions:
To create an Image from an EBS-based Instance
Select an instance that shows "ebs" in the Root Device Type field, click Instance Actions and select Create Image.
The confirmation dialog box appears.
I can successfully SSH into the server and it appears ok.
I can't bring up the site my browser.
Any thoughts?
Comments
re: Creating an AMI Problem
Here's a couple things to look for:
Is port 80 open in the Security Group that this instance is using?
We move Apache to port 81 so that Varnish can use port 80. You can determine if this is an apache or varnish is by trying to view the site on port 81 (https://yoursite:81/). if you can view the site, then it's a varnish error and you might try restarting varnish. If not, try restarting apache.
Hope this helps,
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Ports
Hey Greg,
You're spot on with this one. If I set the tcp protocol to port 81 then I can bring up my site at http://mysite.com:81.
I do get this error message in Pressflow in the admin section:
warning: socket_connect(): unable to connect [111]: Connection refused in /var/www/sites/all/modules/varnish/varnish.module on line 192.
Would you mind sharing what the correct port configuration settings are for Mercury? Somehow I missed that earlier.
Thank you!
Ryan
Unable to restart varnish
I tried to restart Varnish and got the following error:
$ sudo /etc/init.d/varnish restart* Stopping HTTP accelerator
...fail!
* Starting HTTP accelerator
...fail!
Error: (-sfile) "/var/lib/varnish/pressflow/varnish_storage.bin" does not exist and could not be created
re: Unable to restart varnish
Did you run /etc/mercury/init.sh or restart the instance (which causes /etc/mercury/init.sh ti run)? On an AWS server, this places varnish and mysql files into /mnt. That's a good thing on instance-boot servers, but not so much on EBS-boot servers (see http://groups.drupal.org/node/73648).
If /var/lib/varnish is symlinked to /mnt/varnish/lib, and /mnt/varnish/lib deosn't exist. varnish would be unable to start. You might try removing the symlink and recreating the /var/lib/varnish dir with varnish as owner and group.
If that fixes it, then I would also look at mysql to make sure it's not affected by the same issue.
Hope this helps,
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Symlinks to nonexistent directories
You're right. There were symlinks in var/lib to /mnt/varnish/lib and /mnt/mysql/lib, but neither of these existed in /mnt. I've deleted the symlinks and created var/lib/varnish and var/lib/mysql and set owner and group for var/lib/varnish to varnish and var/lib/mysql to mysql.
Unfortunately, I get the same error message when trying to restart varnish:
sudo /etc/init.d/varnish restart* Stopping HTTP accelerator
...fail!
* Starting HTTP accelerator
...fail!
Error: (-sfile) "/var/lib/varnish/pressflow/varnish_storage.bin" does not exist and could not be created
Do I need to create subdirectories in /var/lib/varnish?
So, should I take it as a general rule of thumb to never restart the instance?
re: Symlinks to nonexistent directories
Once the /etc/mercury/init.sh file has run, it won't run again (even on a reboot) unless the incep file is deleted from the /etc/mercury dir.
It looks like varnish would like you to create a /var/lib/varnish/pressflow/ dir - i would also give it varnish owner and group.
Hope this helps,
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Varnish working now, but not mysql
Ok, I created the /var/lib/varnish/pressflow/ and was able to get varnish to restart successfully.
I can't get mysql to restart. After issuing "sudo restart mysql" it hangs. I waited a good 20 to 30 min.
The site is still down. I get the Site off-line page from Pressflow.
The site is currently not available due to technical problems. Please try again later. Thank you for your understanding.Does rebooting the server from the AWS console move varnish and mysql files into /mnt? If not, what caused this?
Thanks
re: Varnish working now, but not mysql
Mercury is designed to do some configuration of the server either by running the /etc/mercury/init.sh script or on first boot (which runs the /etc/mercury/init.sh script). As long as the /etc/mercury/incep file is there, init.sh will no longer run on a reboot or even is started by hand.
One of the things the init.sh script does is check to see if the server is running on an EC2 instance. If it is, it moves some dir's off of the root filesystem onto /mnt since EC2 instances have only 10GB for root partitions. This includes th varnish and mysql files as they can grow to be quite large.
This limited size (and non-persistence) issue has been addressed by Amazon by creating EBS-boot instances. However, we have yet to figure out a consistent way to identify if an instance is EBS-boot or instance-boot (the traditional EC2 method).
Do you have a /var/lib/mysql dir? Is it owned by mysql?
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
I do have a /var/lib/mysql
I do have a /var/lib/mysql directory (I had to create this after launching the instance from my AMI).
drwxr-xr-x 2 mysql mysql 4096 2010-06-22 19:35 mysqlSo, just wondering what the solution to this problem is? This issue came about from creating an AMI from a snapshot of my working Lucid EBS boot instance. After creating an instance from that AMI it seems like the /var/lib/mysql and /var/lib/varnish dirs got screwed up.
This is the state of /var/lib/ directory after creating a new instance from the AMI I created from my EBS Boot Instance:
Only showing the mysql and varnish directories; there are many others in /var/lib
lrwxrwxrwx 1 root root 14 2010-06-19 02:03 mysql -> /mnt/mysql/liblrwxrwxrwx 1 root root 16 2010-06-19 02:03 varnish -> /mnt/varnish/lib
This is the state of the /mnt/ directory
drwx------ 2 root root 16384 2008-03-12 08:01 lost+founddrwxr-xr-x 3 root root 4096 2010-06-24 02:49 tomcat6
So, it seems like there are symlinks to directories in /mnt/ that don't exist. Strange.
re: I do have a /var/lib/mysql
The AMI you created did not copy the /mnt directory (which is the default behavior) which is why the symlinks point to a a dir that doesn't exists.
What I would do is either:
1) move (on your original EBS boot instance) /mnt/mysql/lib /mnt/varnish/lib back to /var/lib/mysql and /var/lib/varnish
or
2) move the same dirs from /mnt to a separate EBS volume which could be attached to any instance.
Either way, when you fire up a new instance from an AMI, /etc/mercury/init.sh will not re-create the synlinks since the /etc/mercury/incep file should exist from your original instance.
Hope this helps,
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
how to determine if AWS instance is EBS or S3 boot image
The AWS EC2 documentation might help you with this question.
In the EC2 API reference:
Look up the API command entry for:
Under Response Elements look for:
rootDeviceType
Description: Indicates the root device type used by the AMI. The AMI can use an Amazon EBS or instance store root device.
Type: xsd:string
Valid Values: ebs | instance-store
Ancestor: item
Children: None
So if you execute a rootDeviceType</> query and get back "ebs" ... you know the answer.
If it returns "instance-store" then the image the instance was booted from is stored on AWS S3 instead of EBS.
I have seen where PYTHON is supported in Drupal.
If that is true then you really should look into BOTO ( http://code.google.com/p/boto/ ).
Boto is a terrific python language library for AWS EC2, S3, RDS, EBS etc.
Boto is also very cool because it also supports Canonical's Ubuntu Enterprise Cloud (UEC) - https://help.ubuntu.com/community/UEC
For development purposes for AWS you can use your own UEC which you can setup yourself in 20 minutes then you have a private cloud that is highly interoperable with AWS (UEC uses Eucalyptus).
Using the UEC/Eucalyptus tools (Euca2ools) its just simple commands to transfer a local private cloud ami to AWS to be used as an AWS AMI -- or to scale beyond your own hw.
re: how to determine if AWS instance is EBS or S3 boot image
This is very useful information, however it requires the user's AWS key which we don't ask them for.
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Great Writeup on how to build a Drupal website on Amazon EC2
I had stumbled across this quite a while ago while trying to learn a little about Drupal. I used Amazon Web Services (AWS) cloud quite a bit so out of curiousity I read the whole article. It was very complete and covers many advanced AWS topics such as Elastic Block Storage (EBS), snapshots etc.
see: Sunset Lake Software's - Running a Drupal Website on Amazon EC2
Be sure to read the many comments submitted to that writeup as there are some great pieces of information provided by some of those that submitted.
For Drupal developers this article should be a very useful reference to keep handy if you intend to use AWS's public cloud.
Brian
I'm having a very similar problem...
I too am having a varnish problem - this is my first Mercury site. It is on a Linode and running on Debian 5/Lenny.
The site answers on whatever port I specify in Apache, but Varnish will not answer. When I try and restart Varnish it gives the following error:
Starting HTTP accelerator: varnishd failed! /usr/sbin/varnishd: invalid option -- S usage: varnishd [options]
And in Drupal it is saying:
Warning: socket_connect() [function.socket-connect]: unable to connect [111]: Connection refused in _varnish_terminal_run() (line 197 of /srv/www/indie-studio/sites/all/modules/varnish/varnish.module).
Not sure what to do - any help much appreciated.
Got the above fixed. Had to
Got the above fixed. Had to do with my vcl file mostly. Was using some syntax that I guess doesn't work anymore.
For newbies, just restart varnish with: /etc/init.d/varnish restart
It will give error messages that will help you get to the bottom of the problems. Look for folders it wants or command lines in config files it doesn't like (typically your /etc/default/varnish and your /etc/varnish/YOURFILENAME.vcl).