I've tried to go through the groups and forums to glean as much information as I could prior to posting this. I fall into the category of someone who is fairly familiar with drupal and mildly familiar with server setups. There is a wealth information on Performance and tuning, but it leaves the novice without a clear path of how to scale. I though I would post my setup here along with my confusion, in hopes that it will help other individuals in the same boat.
My Current Setup
I currently have 400K nodes and 100 users
I hope to be able to scale to 100k + users 5M Nodes
I'm using Domain Access for different "affiliates" (over multisite because some data needs to be shared across sites)**
I am currently using drupal 6.2 on Media Temple (have read that virtuoso really slows things down)
I have somewhere in the neighborhood of 60 modules enabled
I expect a 30% / 70% ratio of authenticated to anonymous users
**not sure how this will effect performance, originally contemplated using multisite to split up the number of nodes for performance. Domain access lets you change table prefix per domain, whiich I think may help performance some.
Path To Scale?
Boost and Memcache
Memcach and Varnish
Pressflow? (im a confused if pressflow is just drupal with some code mods, with these modules pre-configured or something different)
Pantheon ? (im also confused about the difference between Pantheon and Murcury ... is pantheon a service or a tuned customized version of drupal)
Migration to Rackspace - (deploy files on CDN and be able to create a new "instance" on demand
Add load balance / separate DB server
Some Resources
Drupal Documentation On Performance: http://drupal.org/node/627252
Rackspace / Drupal on the cloud http://www.rackspace.com/cloud/blog/2010/05/25/deploying-drupal-in-the-c...
Optimizing Drupla on Rackspace http://www.rackspace.com/cloud/blog/2010/02/02/optimizing-your-drupal-si...
Patheon http://getpantheon.com
Also, I'm leaning toward rackspace just because it seems like the feedback has been a tiny bit better, the Amazon setup seems equally feasible (although maybe a tad more complicated)
Thanks allot in advance
Comments
Definitely recommending
Definitely recommending Patheon AMI on AWS EC2 as described here: http://groups.drupal.org/node/70268 You will have an optimized Drupal (=Pressflow) and if you ever need even more performance, you can scale on that platform - adding read-only mirror databases (suported by Pressflow natively), AWS beanstalk, etc.
---
Tomáš J. Fülöpp
http://twitter.com/vacilandois
Greetings
first off, let me clarify one thing. Performance/Scalability is not free. But fortunately, drupal community is rich with genius and helping people. So this would be relatively pleasure experience.
Secondly, Pressflow is forked distribution of Drupal that eliminates any anonymous user session hence saving few 100 milli seconds right there and improving performance for ANONYMOUS users. It also helps integrate external caching techs like memcache, APC etc which is not possible with plain Drupal (unless offcourse if you apply related patches).
TIPS:
Before you look into these options I highly encourage you look into your site for any refactoring of code if possible. Run benchmark tests on your site to see where does your website spend most of the time. If you install views, panels (expensive modules in terms of SQL and memory), you need to find whether or not you can bring down the query times. This can greatly increase performance right off the bat.
If you have custom modules then bring all form alters in one module. If your website doesnt have complex usage of CCK then uninstall CCK and hard-code custom field programatically to make custom node types.
Golden rule: Never have more than 45 modules. If you do have you need to refactor and get ready for some dirty job.
Anyways the point is, when sites become big enough to handle, extreme and smart measures must be taken. Coming to external cache part.
For anonymous users you have several options.
1. Boost module (plug and play kind off)
2. Varnish (bit advanced but useful reverse proxy server)
3. Nginx (also a reverse proxy server and best option if you are really thinking of scalibity /\ nginx.org)
4. Combine Nginx +Varnish (not required and can be complex)
For Authenticated user experience.
1. Authcache module(super advanced)
2. APC (must have)
3. Memcache (also must)
4. SQL Query cache (advanced)
Hosting
I have heard some bad things about media temple. Dont get me wrong but if you are looking for serious benefits you can think of slicehost.com 5/5 (Part of rackspace company)or linode.com or omega8.cc VPS
As I mentioned earlier scalability and performance is not free and you need to spend both time and money for flawless user experience.
You can save a lot of money by doing above tweaks and avoid CDN all together. Remember if your users come from the other side of the planet then add 500ms - 1000ms overhead (this notion is wrt eliminating CDN, I mentioned above). Remaining is all upto you how smartly and efficiently you can configure your server, MySQL and PHP. Make sure after running your benchmark you properly configure your mysql and php memory. Because if your php starts swapping memory, then you lose time. Use InnoDB rather than MyISAM.
Remember even CDN is only useful for anonymous users. Once you have more authenticated users, they will all hit your actual servers and not CDN, unless you do some heavy CDN integration with akamai/amazon/rackspace/voxel.
Hardware: 4GB RAM, 1 quad core or 2 core 2 duo, SSD or 15k RPM HDD, 100mbps ethernet connection. You are good to go for a million views / day for less than $250/month .. enjoy
****Dont know much about pantheon and mercury
Thanks
I appreciate the feedback thus far, thanks! I do have a fundamental understanding of all the principles that you are talking about, and that much of the advantage of caching is lost on authenticated users. I also understand that nothing is completely free weather it translates into dollars or time. I suppose what I'm trying to do is have some sort of plan on how to potentially scale if the need is there. To me it seems like the time to figure that out is as early as possible. To a novice programmer cck and views are such powerful tools that they seem almost indispensable, however I have begun to be much more conscious of the performance hit they incur when used in certain ways.
Also, just as an exercise I installed mercury on amazon EC2 today and was really impressed with the performance straight out of the gate. It's a little tedious for those of us that are not as familiar with the command line, but I suppose that is part of the learning process. Again, thanks for the feedback having Drupal and the Drupal community is an amazing resource ..
** if anyone is interested, amazon gives you a free (small) Ec2 instance to mess around with for period of time, you don't get charged unless you go over the usage criteria (which is nearly impossible if you are just testing it).
Performance Modules
List of modules that will benefit your site, that are fairly easy to install. And yes this list is VERY biased.
All Users:
http://drupal.org/project/dbtuner - Makes fixing CCK/Views database queries as simple as clicking a checkbox. Use the latest dev.
http://drupal.org/project/imageinfo_cache - This cut our page load times in half if not more. We have a lot of pictures so your perceived gains may vary.
http://drupal.org/project/advagg - Still under HEAVY development but it has some nice advantages when compared to cores CSS/JS aggregation.
http://drupal.org/project/memcache - Use this, it's a good decision. Use for cache tables, session & locking.Stick with Cache Router as this comes with Mercury.Anonymous Users:
http://drupal.org/project/views_javascript_random - Let your random views be cached.
http://drupal.org/project/boost - Easy to get up and running. Use the latest dev.Stick with varnish as this comes with Mercury.http://drupal.org/project/ajaxblocks - Load a block via ajax, allowing for full page caching with blocks that get updated info from the database.
Hey man - I'm launching a new
Hey man - I'm launching a new Project Mercury based site on a Linode, and your links look really interesting. Do you have any updates on any of the modules? I'm of course on Drupal 6.x.
All of my users will be authenticated users. It is a niche social networking site, so the only thing that non-authenticated users will be doing is viewing the home page and registering.
I'm really interested in the imageinfo_cache module. My site will have a lot of images - is using the latest dev release still the way to go?
THANKS.
@dereckd
Hey,
I totally understand your point. But at the same time scaling is a lot about how you build your site in the first place. It really depends on your architecture. Views module now offers caching facility which should be considered without a second thought. Consider caching at every step of your web architecture. Whether you are using view, panels, cck or anything else, implement caching.
Anyways so you have started using amazon EC2. Ask them if they provide load balancing. Load balancing is a way in which multiple CDN servers are configured by amazon, to spread the requests coming to your server by auth & anon users from across the planet. And I am quite sure it is NOT going to be cheap. Also ask them several questions regarding what hardware grade they are using. For instance, the cache memory usually resides in your RAM, so if your amazon instance is using XGB 1333Mhz GDDR3 RAM, it will do phenomenal job at serving requests. Faster the processor (like for ex intel i7), lesser the time to do SQL math, etc
Spend some time trying to find right modules listed below by mikeytown2, mshmsh5000 and several other places in this group, to help meet your requirements. And share your problems here so anyone here, with solutions, can help you out too.
And yes its all learning, experience, failure and success.
Good luck
Number of modules
@denny84: We have Pressflow apps spiking at 18+ million page views per day with well over a hundred modules enabled. Much of this content is served by CDNs, but the point is there's no hard limit to the number of modules.
@dereckd: I think your number (60) would probably be consider a moderate amount by most people, but of course it also depends on the complexity of those 60 modules.
denny84's general point is valuable: in general, the bigger your code base, the more expensive your bootstrap.
The "path to scale" question will have a different answer depending on your team, resources, and the app itself.
Cache infrastructure
Certainly, hardware and cache infrastructure make a huge difference. Having memcache in place and taking load off your database generally makes a massive difference for both authenticated and anonymous users. Having ample memory for the all the app-related processes (Pantheon will include httpd, mysqld, varnishd, memcached, APC) is a requirement that you can best meet by monitoring the situation and tuning over time.
It's very helpful, for instance, to monitor the health of the memcached bins. Cache Router has some limitations with this -- at least did in the last version of the module I enabled -- in that it doesn't display info on all of the enabled instances. But you can just look at the numbers yourself. It's a great way to get more into the guts of memcached.
More here: http://code.google.com/p/memcached/wiki/NewServerMaint
It's also important to make sure that APC has the memory it needs to cache all of the PHP that your app loads. APC ships with a monitoring script that you can load from your web app to view APC health.
http://www.electrictoolbox.com/apc-php-cache-information/
Code efficiency
One of the benefits of Pressflow 6 is that it strips out a lot of Drupal core code that's meant to provide compatibility with older versions of PHP. Similarly, you can look at your own code for inefficiency. This quickly leaves the "novice" level of performance strategy, but the considerations can still be at your fingertips: are you loading more code than you need to? Are you making unnecessary (and, worse, non-cacheable) queries?
Using a code profiler such as Xdebug or XHProf can help get a more fine-grained view of what's going on, but the simpler first steps are to keep a careful eye on the app's Watchdog log and httpd's error log. Sites with too many errors can bog down under traffic pressure, because the sheer Watchdog load, if you're logging to the database, can clog the pipes or even overload mysqld.
Testing
Being able to pound new code in a non-production environment almost always leads to great joy. The two simplest command-line tools are ab and siege. ab is packaged with Apache HTTP. Siege is readily available via most package management tools (e.g., apt-get install siege). In either case, it's handy to be able to compare performance with and without certain features or modules enabled by hammering a URL on the command line.
CDNs
As you said in the OP, offloading content to CDNs tends to yield several benefits: you take load off your app server(s); the user can load content in parallel from multiple locations; and the user may load content from a server geographically closer than your app server(s).
If you start to go this route, make sure to look at http://drupal.org/project/cdn as an integration tool. If Akamai is in your budget, the Akamai module is helpful.
SESSION side-effects
If you migrate an existing Drupal app into a Pantheon install, you may notice that some features for anonymous users don't translate immediately. As denny84 noted, Pressflow doesn't instantiate sessions for anonymous users (neither does D7). This, combined with Varnish serving anonymous content, can cause some session-based content to disappear (e.g., shopping carts, polls, quizzes). It may save a lot of time to do an audit of your app to determine whether any features assume $_SESSION for anonymous users.
That's why mikeytown2 recommended Ajax Blocks, because you can specify that certain parts of the page bypass cache. Similar is Ajaxify Regions.
(Side note: Pantheon and Mercury refer to the same project.)
@mshmsh5000
Sure, pressflow does all the magic as long as its serving anonymous users. Once auth users start logging-in then all or some of those 100 modules start pounding the server. And then we have to seek help from higher memory, to save all the cached compiled php code.
I have experienced problems, specially with auth users and my ab tests fail terribly. So yea, high performance hardware is really the last choice.
a couple 2bits articles might
a couple 2bits articles might help
http://2bits.com/drupal-performance/presentation-34-million-page-views-d...
http://2bits.com/articles/presentation-drupal-backend-performance-optimi...