Pantheon
Pantheon is an open-source effort to create reusable, common, stable, kickass stacks for running Drupal. It began as an effort targeted to Amazon EC2, but really wants to have its stacks run anywhere/everywhere Drupal is found. See also:
Mounting the persistant volume in EC2 best practice
I'm trying to figure out the correct way to mount the ebs volume for persistant storage.
What do you guys do?
I find myself mucking around moving /var around?
do you simply mount on /vol and send symlinks from /var/lib/mysql and /var/www?
Do these symlinks persist after a reboot?
Just trying to brainstorm about the steps you do to attach a volume immediately after provisioning mercury.
Lior
Mercury 0.8-Beta released
We are please to announce the 0.8-Beta release of Mercury AMIs for AWS. Included with this release:
- Removed the predefined admin user from drupal. Now the first login defines the admin user
- Moved drupal cron from the root crontab to /etc/cron.d/drupal for more flexibilty
- Fixed as postifix misconfiguration bug
- Now using symlinks to store mysql, varnish (and now) tomcat data to /mnt rather than moving the actual directories
- A new script to configure varnish, php, tomcat and apc memory usage based on system RAM
Consistent EBS Snapshots for RAID arrays with Mercury
Found some helpful code at Backtype which I've re-written to handle Mercury's defaults and with the assumption you've followed my tutorial on EBS+RAID.
ec2-consistent-snapshot --aws-access-key-id KEY --aws-secret-access-key SECRET --region us-east-1 --mysql-username USER --mysql-password PASS --xfs-filesystem /vol vol-aaaa vol-bbbb vol-cccc vol-ddddSetting up EBS as a RAID
Very similar to setting up an instance with a single EBS drive, you can also fairly easily do a RAID configuration. This leads to better performance usually and higher reliability.
Step 1 - Collect the necessary data
Note the following:
- instance ID
- region (example: us-east-1). Also be sure to note the letter following it.
- access key ID
- secret access key.
Step 2 - Create (Provision) the Volumes
Re: Setting up EBS for Mercury [Beta]
Apologies for the placement of this 'bug report' but I don't have time to correctly change the wiki page here:
http://groups.drupal.org/node/33092
It describes getting mercury running from EBS, but I think that you'll find the latest versions of mercury get mysql to read its data from /mnt/mysql not the usual /var/lib/mysql so the copying and mounting doesn't actually put the database in EBS.
I would research and correct the post properly, but I don't have time at the moment.
Pantheon image downloads?
Hey guys,
First I have to say that this project looks pretty awesome for minimizing setup time on new drupal hosts. I can't wait to give it a go. Question though, is there somewhere I can download the server images? I'm particularly interested in giving them a spin on both vmware and xen. I saw the roll-your-own guide for project mercury which will allow me to get started without a server image, is there something similar for Vulcan?
Thanks.
New Mercury Performance Metrics: Logged-in Users
While Mercury has previously demonstrated the raw power of Varnish to radically accelerate the delivery of cached pages with Pressflow, that's only part of the battle for high performance Drupal. Cached pages need to be generated at least once, and logged in users will be bypassing Varnish when making requests. While it's nice to know you're covered for massive traffic spikes, what about baseline load?
Well, we want to answer that question too. Keep in mind that depending on your module/theme stack, your mileage may vary. A lot. Performance tuning a complex Drupal site for logged-in traffic requires sustained diligence at the application level — code profiling, slow query log review, innovative app-specific caches, etc — as well as hardware and system optimization. However, to set some baseline expectations, we used Jacob Singh's greate starter test suite and checked out a Mercury install on a 512MB Slicehost VPS. The results are quite encouraging.
Mercury 0.71-Beta Released
Mercury 0.71-Beta has been released with small bugfixes (see https://bugs.launchpad.net/projectmercury for details).
AMI IDs are:
US 32-bit: ami-bd7c9fd4 - chapter3-storage/PANTHEON-pressflow-mercury-0.71-Beta.manifest.xml
US 64-bit: ami-b17c9fd8 - chapter3-storage/PANTHEON-pressflow-mercury64-0.71-Beta.manifest.xml
EU 32-bit: ami-8f2c07fb - chapter3-storage-europe/PANTHEON-pressflow-mercury-0.71-Beta.manifest.xml
EU 64-bit: ami-73230807 - chapter3-storage-europe/PANTHEON-pressflow-mercury64-0.71-Beta.manifest.xml
User-Data integration with Project Mercury
I've been working on better integration with EC2 user-data for Project Mercury. The goal is to allow users to specify configuration options when they launch a Mercury instance to simplify the process of manual configuration.
For example if you'd like the EC2 instance to receive an Elastic IP address you'd specify:
IP="new"
Or an existing elastic IP address:
IP="123.123.123.123"
Setting up EBS for Mercury [Beta]
Once you've followed the steps outlined for installing your site on Mercury you'll then want to take your site to the next level using Elastic Block Store. This is going to keep your MySQL database(s) persistent should your server instance crash. Also make it easy to replicate data for testing etc.
There's a good generic Amazon doc page on this here.
Step 1 - Collect the necessary data
Note the following:
<
ul>
Project Mercury Beta!
With great pride, and after six alpha-level releases, I'm announcing of our seventh iteration on the Project Mercury stack, finally baked enough to call "beta".
At this point, we know that many people are using the Mercury EC2 image in production environments, and we've tuned this release conservatively to prevent it from breaking down under heavy load. We've also verified that the stack will work under a resource-constrained VPS (e.g. one with 1/4th the RAM of a small EC2 image), which gives us more confident that this configuration is stable. We also have a kickass logo:
Installing Your Drupal Project On Project Mercury
With the release of Project Mercury BETA, I figured it was probably high time I answered the #1 request for documenteation: "how do I put my existing Drupal site onto Mercury?" This wiki has step-by-step instructions for a few ways of handling this. Your mileage may vary of course depending on what all your site does, but this should at the very least give you a good orientation to get started.
Additional testing of Mercury with 2GB and 512MB RAM
My name is Greg Coit, sysadmin for Chapter 3 and I've been helping with Mercury development and testing.
We wanted a get a quick idea of how hard we could push mercury under more "real world" circumstances, so I combined siege and ab to generate a broad spectrum of hits. ab (short for apache benchmark and part of the apache2-utils package) allows you to generate a very large number of hits on one url, while siege (a perl script which comes in a self-titled debian/ubuntu package) lets you spread the hits across many urls, most of which won't be cached. This mixed-load is a much more nuanced and accurate way of looking at performance than peak throughput on a single url.
EC2 Offering "MySQL Cloud"
Today Amazon announced a provided service for hosting MySQL databases in the cloud: Amazon Relational Database Service. It starts at $0.11/hr, plus $0.10 per GB/month of storage and $0.10 per million I/O requests. The real interesting question is how these will perform. The instances come in many flavors:
- Small: 1.7 GB memory, 1 virtual core with 1 ECU. $0.11/hr
- Large: 7.5 GB memory, 2 virtual cores with 2 ECUs each. $0.44/hr
- Extra Large: 15 GB of memory, 4 virtual cores with 2 ECUs each. $0.88/hr
- Double Extra Large: 34 GB of memory, 4 virtual cores with 3,25 ECUs each. $1.55/hr
- Quadruple Extra Large: 68 GB of memory, 8 virtual cores with 3.25 ECUs each. $3.10/hr
That "Quadrupal XL" is pretty big! I'll be investigating this for Pantheon, but anyone else with experience/info, let's figure out how good these things really are!
Initial Mercury Results From 512MB VPS deploy
Just a heads up; as we move towards more and more stable builds of the Mercury stack, we are starting to look at deploying it on other infrastructure besides EC2. This week, we set it up and tuned for a modest (512MB of ram) VPS. These tests were successful. We were able to simulate a mix of non-cached traffic along side the simple ApacheBench battering, and the system held up well, even without gigabytes of ram to support it.
Mail settings on default mercury
Hi currently when you install mercury you get sendmail as the MTA and I think it's configured to be for local delivery.
I think postfix or my choice - exim are a more fitting (easier to manage and configure).
I don't want to open a holy war but we need an MTA preconfigured to act as an Internet site and not a desktop.
just my 0.02 cents..
Lior
Pantheon Vulcan?
Hi all,
Is anyone using Vulcan? I've installed the EC2 instance, but can't find any documentation on configuration for selenium, hudson, etc. I've used selenium on our other boxes before so I'm familiar with that.
What is packaged into the Vulcan instance? Is the server just a skeleton or are there packages set up already? Is this ready to go or just an instance I'd build manually? What are the advantage to using the Vulcan EC2 instance?
Thanks
Project Mercury Alpha 6: Now With Solr!
I'm happy to announce the 0.6 Alpha release of the Mercury AMI, now including ApacheSolr as the search backend! This is the last piece of major infrastructure we want to integrate into the stack for scalability purposes. You can now move from a single-server install based on Mercury to a best-practice vertically scaled architecture with separate hardware to run front-end cache, application, back-end cache, search and database!
The quickest way to find it is by searching Amazon EC for "Pantheon" or "Mercury". The manifest path for the latest release (in 32bit and 64 bit flavors) is:
chapter3-storage/PANTHEON-pressflow-mercury-alpha-6.manifest.xmlchapter3-storage/PANTHEON-pressflow-mercury64-alpha-6.2.manifest.xml(back!)
If you'd like to "roll your own" we've updated the wiki instructions page with a new set of instructions for getting Solr up and running as part of the process. Feel free to improve that documentation, as it's definitely a community process.
This will likely be one of the last releases before we move the project into the Beta phase, at which point we'll be focusing on fine tuning and stability as well as portabilty onto non EC2 systems moreso than new features. If you have ideas for additional things you'd like to see integrated in the stack, please chime in. We're also going to be documenting real-world "how to" use-cases — e.g. "how do I put my existing site on Mercury" in user-friendly detail — so stay tuned for that.
As always, let us know what you think of the release, what you'd like to see in future iterations, and how your experience is in using the stack. There's plenty more to come.
Update Pantheon Aegir AMI
Can the maintainers of the pantheon AMIs please update the Aegir AMIs?
I'm quite eager to use it, but would like it to be using the latest version please.
Thanks!
