high performance

Use of Drupal is exploding here at UC Berkeley. The campus doesn't have a good solution for Drupal hosting in place. For the past year it has been my vision to create an environment to do Drupal hosting the right way. I have done a lot of work an have some solutions in place. Now we want to get our plans reviewed reviewed by a LAMP/Drupal systems expert. Does anyone have recommendations for a systems consultant?

More about the advice we are looking for:

While Mercury has previously demonstrated the raw power of Varnish to radically accelerate the delivery of cached pages with Pressflow, that's only part of the battle for high performance Drupal. Cached pages need to be generated at least once, and logged in users will be bypassing Varnish when making requests. While it's nice to know you're covered for massive traffic spikes, what about baseline load?

Well, we want to answer that question too. Keep in mind that depending on your module/theme stack, your mileage may vary. A lot. Performance tuning a complex Drupal site for logged-in traffic requires sustained diligence at the application level — code profiling, slow query log review, innovative app-specific caches, etc — as well as hardware and system optimization. However, to set some baseline expectations, we used Jacob Singh's greate starter test suite and checked out a Mercury install on a 512MB Slicehost VPS. The results are quite encouraging.

Mercury 0.71-Beta has been released with small bugfixes (see https://bugs.launchpad.net/projectmercury for details).

AMI IDs are:

US 32-bit: ami-bd7c9fd4 - chapter3-storage/PANTHEON-pressflow-mercury-0.71-Beta.manifest.xml
US 64-bit: ami-b17c9fd8 - chapter3-storage/PANTHEON-pressflow-mercury64-0.71-Beta.manifest.xml
EU 32-bit: ami-8f2c07fb - chapter3-storage-europe/PANTHEON-pressflow-mercury-0.71-Beta.manifest.xml
EU 64-bit: ami-73230807 - chapter3-storage-europe/PANTHEON-pressflow-mercury64-0.71-Beta.manifest.xml

With great pride, and after six alpha-level releases, I'm announcing of our seventh iteration on the Project Mercury stack, finally baked enough to call "beta".

At this point, we know that many people are using the Mercury EC2 image in production environments, and we've tuned this release conservatively to prevent it from breaking down under heavy load. We've also verified that the stack will work under a resource-constrained VPS (e.g. one with 1/4th the RAM of a small EC2 image), which gives us more confident that this configuration is stable. We also have a kickass logo:

I'm happy to announce the 0.6 Alpha release of the Mercury AMI, now including ApacheSolr as the search backend! This is the last piece of major infrastructure we want to integrate into the stack for scalability purposes. You can now move from a single-server install based on Mercury to a best-practice vertically scaled architecture with separate hardware to run front-end cache, application, back-end cache, search and database!

The quickest way to find it is by searching Amazon EC for "Pantheon" or "Mercury". The manifest path for the latest release (in 32bit and 64 bit flavors) is:

• chapter3-storage/PANTHEON-pressflow-mercury-alpha-6.manifest.xml
• chapter3-storage/PANTHEON-pressflow-mercury64-alpha-6.2.manifest.xml (back!)

If you'd like to "roll your own" we've updated the wiki instructions page with a new set of instructions for getting Solr up and running as part of the process. Feel free to improve that documentation, as it's definitely a community process.

This will likely be one of the last releases before we move the project into the Beta phase, at which point we'll be focusing on fine tuning and stability as well as portabilty onto non EC2 systems moreso than new features. If you have ideas for additional things you'd like to see integrated in the stack, please chime in. We're also going to be documenting real-world "how to" use-cases — e.g. "how do I put my existing site on Mercury" in user-friendly detail — so stay tuned for that.

As always, let us know what you think of the release, what you'd like to see in future iterations, and how your experience is in using the stack. There's plenty more to come.

Just wanted to let everyone know we're hard at work on a Beta release of Project Mercury. Currently we are tuning Varnish further, working to package Apache Solr on board as the search backend, and testing the whole system for bugs. We will likely issue one more alpha release (0.6) before we get to a Beta, but I want to have that launch by the end of the month if possible.

The high level road map:

• Package Solr
• Improve default Varnish config
• Thoroughly document "putting your site on mercury"
• Resolve lingering bugs (e.g. cron.php issues)

Once we hit Beta we'll be pushing for wider adoption, which means pursuing alternate non-Amazon hosting options. Currently we're looking at alternate cloud providers, VPS partnerships, and other packaging options like VM images, etc. So stay tuned, and let us know what else you'd like to see us look at including early in the Beta cycle!

mod_xsendfile

mod_xsendfile is a small Apache2 module that processes X-SENDFILE headers registered by the original output handler.

If it encounters the presence of such header it will discard all output and send the file specified by that header instead using Apache internals including all optimizations like caching-headers and sendfile or mmap if configured.

It is useful for processing script-output of e.g. php, perl or any cgi.
Referance from http://tn123.ath.cx/mod_xsendfile/

I was looking for frontend tuning for faster page load. I found some very good technics, almost all already improved by drupal, but I can't figure out how to load javascripts in the button of the page. As you can see Javascript is loaded on the top of each page, while that happens page loading is blocked to other things like loading images,flash and also cause a gap while processing that code and restart loading the rest of the page...

I think these urls are good guides for improving front-end performance:
http://code.google.com/intl/en/speed/page-speed/docs/rules_intro.html

I would like to test some caching configuration for my site. Could someone recommend me some tools to test the effectiveness of the site?

Also, if there are known configurations that work for a site with only anonymous users, please let me know! (currently using eAccelerator and Boost).

Thanks!

I just wanted to share some technical information about a very different caching scheme, in hopes that someone would be able to make use of the information for Drupal.