Looking for some technical experiences with Drupal

Posted by glove on March 30, 2007 at 4:57pm

Would anyone be willing to share their technical experiences with me, about how you set up Drupal in your production environments? We're extremely interested in Drupal for its "community plumbing", but the engineering group here has some very valid concerns about how it scales with a large number of pageviews and users.

Comments

It scales

Posted by jlambert on March 30, 2007 at 5:02pm

You can scale it with the right topology. We've gotten more than 15m pvs per day out of Drupal 4.7 before (8way, 24GB ram, mind you, but it can scale).

The basic methodology behind it is cache, cache, cache. Memcached + Wackamole or haproxy or Zeus LB + performance database (postgres m+m or mysql cluster) + split web server + squid or squid equivalent (There are a few) + lots of memory and reasonable horse power is the basic formula. Then you need to load test, code profile, and tune your systems to success.

Jonathan Lambert

Any core hacks to make that happen?

Posted by glove on March 30, 2007 at 5:09pm

2 questions:

Did you have to make any hacks to Drupal's core to achieve that scale? Our preference would be to use system tuning and modules (home-built or distributed), so that we don't box ourselves in the corner when it comes to upgrades/patches.
Are you using Drupal's authentication system? Or Drupal's caching system?

Gary

I cannot answer that

Posted by jlambert on March 30, 2007 at 5:33pm

I can't talk too much about that specific implementation unfortunately.

However, no not really.

Yes. Both. The cache system hasn't given us dramatic results, but you should use it as it does help.

Jonathan

Throwing hardware at it

Posted by yelvington on March 30, 2007 at 8:27pm

We have one site on an externally hosted server, but Savannah and everything bloglike for all our other papers is served by a cluster of three webservers and one database server. Drupal is designed for this sort of configuration, so it no patches are required.

Each box is a dual-CPU Intel running Linux. The webservers are fronted by an allocator. One webserver is the "master," with r/w access to the filesystem (for uploaded images, etc.) and the others mount the htdocs-containing disk image remotely, so the allocator has to route PUT requests only to the right physical server, but there are no other system tricks.

I recall that early in the process the tech team tripped over some bugs in PHP/APC that required periodic restarts of the Apache process to clean up memory leaks. I don't know if that is still the case.

Savannah's section fronts are cached in the filesystem using some code I've never seen and don't understand. Ken might know more.

Re: Throwing hardware at it

Posted by dpauken on March 30, 2007 at 8:45pm

Throw the best hardware you can afford at it.

I've run into issues recently, the hardware we're running on isn't built for much 'heavy lifting' or handling the volume of request to the database (needless to say, we'll be fixing that issue).

I'm not sure if my problem was related to a memory leak in the system, but it certainly wasn't at the forefront of my mind when we deployed drupal.