Posted by Greg Coit on February 12, 2010 at 2:01am
Mercury 1.0 is almost ready - in fact we hope to have it out before the end of the weekend.
We wanted to give folks a preview of the instructrions to create their own Mercury 1.0 (note: nothing is written in sand until Mercury 1.0 is released - we still have testing to do):
2/26/10: We've released 1.0RC for testing and comments (see http://groups.drupal.org/node/50153#comment-145523 below).
2/13/10: Updated and moved to a wiki at http://groups.drupal.org/node/50408
Comments
Thanks for the preview. Its
Thanks for the preview. Its nice to see the bcfg2 stuff in there too very cool!
^_^
Would this work on small VPS
Hi - Great work so far I am following this project with great interest. I have a few questions below I hope you don't mind answering:
-I know you have tested this on a 512MB VPS but my current budget and needs means I am using a 256MB VPS (burstable to 512MB). Would Mercury work ok on this server or is it a tad underpowered? Could I skinny it down a bit by using boost ilo varnish and PHP Lucene (or default search) instead of Apache Solr would this be an option?
-Also I was thinking of using Aegir with mercury / pressflow but the install instructions seem different (pressflow is in /var/www not /var/aegir/platforms/pressflow). Can I just tweak these things or would this break the bcfg2 pre config stuff?
-Would it be best to install mercury first then aegir on top of this?
Thanks In Advance,
JamieT
I've not installed Mercury on
I've not installed Mercury on a 256MB server - i think the the answer depends on this size of the site. mercury trades memory for speed via caching and if you site exceeds the 256MB the benefits of Mercury will be lost.
And I've not yet combined Mercury and Aegir - I believe others on this list have. My guess is install Aegir first, then Mercury. There's nothing in the bcfg2 pre-config that will be affected - but much of the paths in the post config will have to be changed as well as some of the config files that bcfg2 manages.
Hope this helps,
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
I've Aegir running
I've Aegir running successfully in mercury .. but I havnt done it in combination with the bcfg2 install script yet. I've gotten Aegir to work both starting with the Mercury EC2 AMI and I currently have Aegir & Mercury running in a "handbuilt" EC2 Ubuntu-Karmic setup. Im going to try it out again over the weekend and see how it goes with bcfg2. I still need to map out whatever conflicts and make sure to keep that in mind for later mercury updates, AFAIKT there should not be too many. Bcfg2 is too cool!
I'd be interested in an
I'd be interested in an update when you get this working. I guess once complete I could update the BCFG2 configs and share. Out of interest did you install pressflow under /var/www as per above instructions or did you install to /var/aegir/platforms as per aegir install?
Thanks In advance,
JamieT
Well I have made the changes
Well I have made the changes to the config files .. mostly changing them to work with Ubuntu 9.10 Karmic. Ive not included Aegir stuff yet.
Ive yet to get the (bcfg2 -vq) command to work .. AFAICT I have things setup proper but when I run that command it hangs for like 6 or so min and then returns...
Server failure: timed outFailed to download probes from bcfg2
timed out
There's not much googling with that error and whats there hasn't enlightened me as to what could be the problem.
Aegir Directory
I run my Aeigr from /vol/aegir its a mounted ebs raid but you could place it anywhere; its the home dir for the Aegir user. The actual pressflow is a platform/s within the aegir directory, but remember the platform for the main aegir site should not be pressflow, just use the standard install script with Aegir. I run aegir latest alpha5, the install is super fast & simple.
I think we could include some aegir steps in bcfg2 but first I need to get bcfg working lol. Im wondering if its because Im using Karmic instead of Jaunty?
BCFG2
I am / was having similar issues with bcfg2 -vq using rackspace cloud server -which is also Karmic. But all of the software did get installed and running until I had a "stupid user" (yes it was me) problem that messed up the install so I will have to start over.
I was pretty unclear how to do some of the config for BCFG... but I have never heard of it before.
Joe Moraca
WebDevGeeks.com
Did you get bcfg2 to work? I
Did you get bcfg2 to work? I have not had any luck with it yet... I dunno maybe there are some additional steps to be taken with an AMI/server before going through the bcfg2 steps.
Does the server need a real hostname or will an ip or amazon supplied address suffice? In my latest test on the first run it seemed to partially get going but had a lot of fails and pretty much didn't install anything, So I changed up the hostname & when I ran it again I got a "Client metadata resolution error" on & all consecutive runs timed out saying could not download probes. Im starting with a fresh ubuntu karmic and assumed that apt-get bcfg2 would sort all needed dependencies , if its not that or a hostname issue im at a loss to what could be mucked. I'll try it again later, its quite annoying though since I can install all the mercury steps manually fine. Still I know bcfg2 will be far superior once I can get it to work; It will be nice to see the 1.0 release and get a look at the bcfg2 configuration and such to see where it is that I went wrong.
See the thread
See the thread dedicated to BCFG2:
http://groups.drupal.org/node/50408
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
Yes those are the
Yes those are the instructions I followed. The only thing I changed in the various bcfg2 files was to use Karmic instead of Jaunty.
Perhaps there is some conflict with the Karmic AMI I used, I'm moving tests back to jaunty to see if I can get the bcfg2 to work on that better, at least it requires much less editing of the standard starting point then I can work in changes from there. I'll give it another whirl later on.
Yup, same issue here
I've been fooling around with doing the Mercury install and get stuck at a couple of places
The server just basically hangs. I have to kill the process.
I've documented my issues in the wiki (http://groups.drupal.org/node/50408) and at http://groups.drupal.org/node/52373#comment-145828.
I've tried it on two different AMIs:
Ubuntu 9.04 Jaunty Server, AMI is: ami-0d729464.
Ubuntu 9.10 Karmic Alestic, ubuntu@ EBS boot, 32 Bit (ami-6743ae0e), EBS Boot
Would love to know what's going on.
re: Yup, same issue here
MyRy,
Please seem my reply to your post here: http://groups.drupal.org/node/52373#comment-145828 - I don't think the bcfg2-server is hanging.
Thanks!
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
re: Did you get bcfg2 to work?
Macrocosm,
bcfg2-server asks for the hostname from the user at bcfg2-init and adds it to /etc/bcfg2. Hence, bcfg2 is unable to authenticate ("Client metadata resolution error") and doesn't work if the hostname is /var/lib/bcfg2/Metadata/clients.xml is changed. The good news is that the amazon supplied private-dns is perfectly fine for bcfg2.
We don't use it for the amis we release since every server created from out amis get a new private-dns, so we set everything to be localhost.
If you need to reset the hostname that bcfg2 uses. the following steps will do the trick:
#kill the bcfg2-server and removed cold config files:
/etc/init.d/bcfg2-server stop
rm -rf /etc/bcfg2* /var/lib/bcfg2
#re-init:
bcfg2-admin init
#save the new clients.xml file and replace the rest from launchpad:
cp /var/lib/bcfg2/Metadata/clients.xml ~/
rm -rf /var/lib/bcfg2/
bzr branch lp:pantheon/bcfg2 /var/lib/bcfg2
mv ~/clients.xml /var/lib/bcfg2/Metadata/
Edit the following files as needed:
/var/lib/bcfg2/Metadata/groups.xml
/var/lib/bcfg2/Metadata/clients.xml
Hope this helps,
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Thanks for the tips I am
Thanks for the tips I am going to test out everything with the new 1.0 asap, it will be nice to see it all in and functional so I can see where I went wrong. I think I just kicked it around too many times.. >_<
Thanks for the reply I guess
Thanks for the reply I guess I will try and install aegir first and then mercury using your directions. I know boost is different to varnish and memcache but would that be a better option with a smaller server. The website isn't going to be big to start but there's no harm in starting out with expansion in mind - if it takes off I can resize the vps and fully utilise mercury then. I'll feedback when I have made progress.
Any thoughts on the above.
TIA,
JamieT
512mb minimum
512mb of ram is the minimum we can recommend for Mercury, and my general experience is that Drupal itself can be problematic with only 256mb. It's just too easy to get sent into the downward spiral of swap.
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
rackspace
I started setting up a rackspace cloud server (1GB) last night - I will use these instructions and let you know if I have any issues. Are there any difference between 32 and 64 bit server installs?
Joe Moraca
WebDevGeeks.com
Joe
joe,
I'm not aware of any differences. We've placed AMD64 and i386 profiles in BCFG2 in case any appear.
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Updated and moved!
These instructions have been updated and moved to a wiki: http://groups.drupal.org/node/50408
Our intent is to keep both http://groups.drupal.org/node/25425 (non-bcfg2 instructions) and http://groups.drupal.org/node/50408 (bcfg2 instructions) up to date with Mercury versions as they are released (though the non-bcfg2 may lag a bit). Mercury 1.0 is still looking like it will be released before the end of the weekend.
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Can you guys please include
Can you guys please include the uploadprogress module as part of AMI, or is that outside of the scope?
You can extend
The idea here is that you can extend. BCFG2 is good about allowing "unmanaged" entities.
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
SSL
Please include a note on enabling SSL using this image. I need to use the Secure Pages module and would like at least a self signed cert for use in testing on initial install.
Thanks!
Out of scope
Mercury 1.0 is all about production, in which case self-signed certs aren't any good. This is a case where you still need to be able to manage things on your own. However, as per the above, the system is designed to support site/project specific development and tayloring.
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
Sorry for the delay
Sorry all for the delay. I had to leave town for a week and was away from the computer.
Obviously, the launchpad paths that I posted to http://groups.drupal.org/node/50408 were incorrect - this has been fixed (the code is now hosted at https://launchpad.net/pantheon). We split bcfg2 and profiles into separate branches allowing you to pull updates straight into your system if you choose (ie: cd /var/www/profiles or /var/lib/bcfg2 and bzr merge --force). This will allow folks to incorporate most of the future changes/updates to Mercury without having to re-install. We intend to use trunk for development and only update bcfg2 and profiles when trunk is stable.
This technique can also be used to update pressflow too (cd /var/www; bzr merge --force).
We also made some other updates to the instructions at http://groups.drupal.org/node/50408 including updating drush to 3.0-Beta1. We'll update the original Mercury install wiki (http://groups.drupal.org/node/25425) as time permits but this is a little more involved.
This time I mean it - Mercury 1.0 will be out by the end of day today in both 32 and 64 bit versions for Ubuntu Jaunty. We do still intend to make Ubuntu Karmic (and maybe Cent-OS), ebs-boot (hopefully) and non-AWS (probably rackspace) versions as time permits - sorry for the wait!!! :)
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Great News!
Glad to hear it, Greg. Thanks for doing this.
Sweet!
You guys rock so much!
Im looking forward to testing the new 1.0 release, and attempting the bcfg2 install again.
Cheers!
Thank you!
I really appreciate updating the instructions. I'm really pretty new to this, so having good instructions really help. Looking forward to trying the install again soon.
Pantheon Mercury 1.0RC released
Hi all,
I can't thank you folks enough for your patience and encouragement!
I'm stoked to announce the release of Pantheon Mercury 1.0! (Well, almost - we're calling it 1.0RC just to play it safe) :)
We've added tons of new features, tweaks and bugfixes - here's a short list:
Full integration with bcfg2 for config file management
Mercury checks for (and incorporates) updates from launchpad on first boot
Updated drush to 1.0-Beta3
Added mercury profile to auto-populate pressflow with mercury-specific settings (varnish, memcache and apachesolr)
Added /usr/local/bin/update_pressflow.sh script to update pressflow from launchpad
Added /usr/local/bin/update_mercury.sh to update mercury config from launchpad (and apply changes using bcfg2)
The last feature is not automatic - we're not controlling your server. It gives you the option (if you wish) to grab mercury updates from our project on launchpad and incorporate them into your server - the only thing we do automatically is grab updates on first boot in case there are bugfixes. We have a development branch of our code (called trunk, of course) on launchpad - the project name is pantheon (we're deprecating projectmercury). Pantheon also hosts stable branches called bcfg2 and profiles which is what the init.sh and /usr/local/bin/update_mercury.sh scripts pull from.
You'll notice that we ask you to wait 2 or 3 minutes after boot before configuring pressflow - it can take bcfg2 a few minutes to spin up on first boot.
The ami's are uploaded and public. Please test these images this weekend - we'd like to see it poked, prodded, and pushed to its limits. Please post this this list any problems/concerns/questions you have! We'll incorporate any needed bugfixes/features and release 1.0 early next week.
ami-99a44bf0 pantheon-amis-us-east/mercury-pressflow-drupal-32-1.0RC.manifest.xml
ami-8da44be4 pantheon-amis-us-east/mercury-pressflow-drupal-64-1.0RC.manifest.xml
ami-bbbd96cf pantheon-amis-eu-west/mercury-pressflow-drupal-32-1.0RC.manifest.xml
ami-a5bd96d1 pantheon-amis-eu-west/mercury-pressflow-drupal-64-1.0RC.manifest.xml
(us-west version coming soon).
Thank you!!!
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Awesome!
You guys rock so much .. testing tonight!
^_^
Detailed change log?
Thanks for your great work.
Is it possible to view a detailed change log on https://launchpad.net/pantheon ? I hunted around the site but couldn't find it.
Agileware, Australian Drupal Developers
http://agileware.com.au
re: Detailed change log?
Justin,
There are 2 options - neither perfect. The bzr revision log for trunk is viewable (http://bazaar.launchpad.net/~pantheon-developers/pantheon/trunk/changes) but that's probably too much detail. We also have a changelog.txt file (http://bazaar.launchpad.net/~pantheon-developers/pantheon/trunk/annotate...) but that's probably not detailed enough. However, between the 2 of them, you can see everything we changed/updated/fixed.
Hope this helps,
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Thanks
Thanks Greg, appreciated. Man, I am pretty impressed by Launchpad - how do you find it for development, packaging etc? Good?
Agileware are an Australian Web Team specialising in Drupal, WordPress and CiviCRM.
Agileware provide development, support and hosting services.
https://agileware.com.au
Launchpad
Very, very good. :)
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Updated EU-WEST AMIs
Had to re-create the EU-WEST bucket - new AMI IDs are:
pantheon-amis-eu-west/mercury-pressflow-drupal-32-1.0RC.manifest.xml ami-ffb2998b
pantheon-amis-eu-west/mercury-pressflow-drupal-64-1.0RC.manifest.xml ami-f3b29987
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Multisite trouble
Hey,
First of all: thanks for the update, it's looking quite professional :).
Anyway, I can't seem to get a multisite working, as soon as I change anything on one site, it is changed on the other site as well. While the nodes are loaded properly, the themes and variables aren't. Do you have any idea for a solution?
Thanks!
Mark
UPDATE: I already found it, for anybody else: I needed to use a unique memcache prefix per site as per instructions on http://drupal.org/project/memcache ("Prefixing").
Maybe it's possible to overcome this by using a unique variable per site (e.g. database-name).
Same problem even when using Memcache prefix per site
I've had the same problem even when using Memcache prefix per site. So I moved to APC instead, problem went away.
Agileware, Australian Drupal Developers
http://agileware.com.au
Yeah
I wound up going back to APC + cacherouter as well because clearing caches would break the theme layer. Memcache is not a happy camper for me; but ive not tested the latest Mercury release yet... cant wait to test the new one!
The other thing thats gotten me for multi-sites is Solr. I've yet to get a working multi-core solr inside mercury; which really is ok since only one of my sites uses solr. Still I think it would be best to release Mercury with multi-site compatibility built-in to start, most people will want at least a couple sites on their box, and even if they only ran one site it wouldn't impede their usage I dont think.
us-west AMIs
us-west AMIs of Mercury 1.0RC are available:
ami-e74a1ba2 pantheon-amis-us-west/mercury-pressflow-drupal-32-1.0RC.manifest.xml
ami-ed4a1ba8 pantheon-amis-us-west/mercury-pressflow-drupal-64-1.0RC.manifest.xml
Thanks!
Greg
--
Greg Coit
Systems Administrator
http://www.chapterthree.com
Great Work!
Thanks for the update. Was able to launch a test site using ami-e74a1ba2 (32bit).
Latest AMIs
I'm getting a bit confused by which one is the latest AMI. I updated the Install wiki to show what I found. Can I suggest we keep that as a source for which is the latest AMI. Alternatively, is it safe to say that if one searches in the AWS console for Mercury, they will get the latest AMI.
Too funny, just also realized that when you change your region in the top left of the AWS console you have access to different AMIs. ;0
re: Latest AMIs
NyRy,
Good idea on listing the AMIs in the wiki and the AMI's you listed look correct. I also think searching for mercury will work (although it's likely that now we've hit 1.0 we'll stop deleting old AMIs when newer version come out).
--
Greg Coit
Systems Administrator
http://www.chapterthree.com