How Much can Drupal Handle?

Helo John,

Someone that has been doing a lot of work with performance is Khalid Bahey from 2bits, I always read the articles on their site and they frequently talk about performance. I recommend in particular this one: http://2bits.com/articles/can-a-drupal-web-site-handle-a-million-page-vi... I can also recommend this excelent article by Wim Leers: http://wimleers.com/article/improving-drupals-page-loading-performance it helped me get from a low D to a high B with Yslow and the steps where simple to follow, although I use the Javascript aggregator module instead of the patch they mention on the article.

Big sites like band or record label sites probably have a cluster of servers or something like that, so it is not entirely up to Drupal if it can handle your visitors, one of the most important thing would be your server setup and you MySQL configuration. I've used things like advanced cache and block cache and they help greatly reduce the amount of MySQL queries a Drupal page has. But advanced cache needs some work right now, the patches are not working properly. I used the path patch and it reduced seriously the amount of queries because drupal_lookup_path was getting cached, because most menu items in Drupal are really a query. I wouldn't use it in a production site though because I some of the hunks failed when I applied the patch.

http://nic.ipwa.net

The company I am working for is maintaining the universal music group site...

There are a couple of millions page views a day and everthing is working fine.

Depends on Server and Database like the guys above me said. For my private projects I am using Lighttpd which is recomend by Dries...

Drupal Rocks !!!

Hi.

It absolutely can scale.

There are folks who specialize in performance and scalability, and we partner with them. Folks like Tag1, who have also made available a book chock full of performance and scalability techniques: http://books.tag1consulting.com/scalability.

We've built high-profile sites like NASCAR's www.roushfenway.com, Bono's www.data.org and more.

I'd be happy to help!

Best,

--D

--
http://www.digital202.com
http://www.wovenlabs.com

There's no doubt that Drupal can scale pretty big. Drupal itself isn't the problem. I used to work for Sony BMG where I managed their multisite install, and now I work for Lifetime. The issue is all in the implementation of hardware. Your bottlenecks are going to be in how much your webserver(s) can handle, how dependent you are on your database (memcache makes this much easier), and how much your database server can handle. The biggest limitation that you may run into is how much MySQL can handle. When you start dealing with sites that have a LOT of content, you're going to lose a lot of performance just in searching the node and user tables. Memcache can help, but you only have so much room in memcache...

If you'd like an extra set of hands, from a consultant experienced with large scale sites, drop me a line, and I'd be happy to help you and your friend design a server architecture that will handle whatever traffic you throw at it.

Mike

for your comments.

I will point my friend to this page so that he can read the thread!

Yes, Drupal Rocks!

First, let me clear up some things from a few of the comments above.

MySQL isn't going to be your limiting factor. MySQL can scale to massive traffic. See Yahoo and Craigslist for examples of that. The issue is that in stock Drupal, you're tied to a single database server, limiting your growth to the performance of that single server. You can add patches that will allow you to split reads and writes to different DB servers, sending all writes to a master server and all the reads (which should outnumber writes 20-1) to a cluster of slave servers. These are the patches Drupal.org uses.

Simply loading records from the node or user tables won't impact performance, no matter how large your data is. The tables are well designed, with intelligent indexes. It's possible that some modules might try and read data from them in ways that hurt performance, but you can quickly find those and either tune the query or the database. Where you're going to hit issues with large data sets is in search. You could possibly run into some lock contention problems as well.

Drupal's built in search is fairly intensive on the DB. You can overcome this by adding a third-party search server, either by crawling your site as a user would or by implementing something like Lucene, Solr, or Sphinx that reads right from the database. You've got to think about user access to content with this approach - these search indexes don't know anything about Drupal's access control.

Most sites have their MySQL databases running MyISAM, which can become an issue for tables that have even moderate numbers of writes. Every write locks the whole table, preventing reads from happening. Moving to InnoDB can help there. InnoDB isn't the right choice for every table, through. Tables that are rarely written to will perform better as MyISAM. MyISAM tables are also smaller. So you need to think over your usage and plan things out.

Your traffic profile is going to seriously affect how you do things. For instance, if you have mainly anonymous usage you can do a lot to reduce the load on Drupal and the DB. Reverse proxies can completely offload the traffic from Drupal entirely. And the memcache patch can ensure that most of your content comes from local memory instead of the DB.

There's so much involved in scaling Drupal (or any other web site) that it's hard to hit all the high points in a forum post....

• Reduce the number of modules you use, paying particular attention to write-heavy modules like Statisitics.

• Cache, cache, cache. Built in cache, CSS aggregation, Block Cache, Memcached

• Use Opcode caches to improve PHP performance.

• Tune the stack -- make Apache perform better, replace apache with Nginx, make sure your DB server is tuned for the traffic and data size

• Optimize your code -- watch the query usage and performace characteristics of third party modules

• Push static content to CDNs or external web servers

We've accomplished some massive scaling on Drupal. We got Macworld Expo to handle 10+ million page views in a single day. We helped a major drug company run tens of thousands of concurrent users. So, yes Drupal can scale.

-- Adam Kalsey, WorkHabit.