I'm in charge of 100 plus drupal sites in use by the Purdue University Professional Writing program. We host sites used by instructors teaching multiple courses in PW as well as this Fall introductory composition. Currently, we are using a single multisite install to host all of these sites, with all the courses running off the main code. We're using Storm On Demand hosting, with their intro level server, which is one core on an intel quad core, with 2gb of ram.
My boss is interested in giving instructors a bit of freedom in the future, as their experiences goes up with the tech, and would like to give them their own install to play around with. Additionally, we're moving towards a model where new instructors will have a version of the teaching site that has been designed in conjunction with their mentors. To do this, we've been looking into different options.
What I am wondering, is if the load of having these 100 plus (150 or s currently) sites running off multiple drupal installs (lets say 6 installs) would problematic for our server. I've read that the more stand-alone installs you have on a server, the more the load since different code is being referenced by different sites, but am not too sure how much of an impact that makes on performance, and was looking for some feedback.
(Additionally, I would add I'm learning the back-end of Drupal in chunks, as things come up and we fix them or deal with them, etc. I got this job in July, and have been striving to get our sites stable, happy, and usable since)
Also, as a bit of explanation: each site is used by an instructor and around 22 students. The sites are often used during class, with multiple classes running at once on our schedule during the week, giving our server periods of concentrated load that cannot be avoided.
V
Comments
I would not see why having
I would not see why having multiple installations of Drupal vs. a single one would make any performance difference. The one exception to this is if you are using a PHP Accelerator. In that case, YES it could make a difference.
On the other hand, you would be creating a maintenance nightmare by creating 6 separate Drupal installations. You would have to perform all maintenance actions 6 times... time consuming.
I assume by PHP accelerator
I assume by PHP accelerator you mean an opcode cache (ala APC). But then it's just a matter of giving the cache a bit more memory if necessary.
And I wouldn't describe 6 code-bases as a maintenance nightmare. You just need to find a balance with the cost of increased management of multiple code-bases vs. the benefits of splitting them (generally the more dissimilar your sites become, the greater the benefit of having multiple code-bases).
I'm guessing that these sites neither have a lot of traffic, or a lot of content, so you biggest issue with performance is going to be either module bloat, or poorly written code. You are more likely to find performance issues in code that hasn't been used by a wide audience (hence not well audited), either because it's custom code, or an uncommon contrib module.
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
opt for one base install
I would go for one base install; and if needed different sites/*/modules dirs that are symlinked to your 6 different install profiles with their collection of modules etc... placing the most common modules (view/cck/pathauto...) in sites/all/modules. I would be more worried about maintenance then performance in your case; you can have multiple webheads but dealing with multiple versions of files on a live site can be a major pain. Same applies to themes.
I would also go for PHP 5.3.3 FPM with Nginx & APC.
http://groups.drupal.org/nginx
Just to give an alternative
Just to give an alternative viewpoint again, based on your sysadmin skills PHP 5.3.3 FPM with Nginx may not be the way to go. It will give you a minimal performance gain at the cost of a fair bit of work. Plus PHP 5.3 will likely break any site more than a few months old. If you already have a functioning webserver, stick with that. If your sysadmin chops are not strong just stick with the standard Apache setup as it's solid in every distribution of every OS.
But do install APC if you haven't already. It brings ~20% performance improvement in page generation time regardless of site size.
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
Thanks for the advice
Thanks for the advice everyone. It is starting to sound like a single install would make the most sense for us to be able to have the easiest time with things. Maintenance is a big issue, since currently I'm tasked with maintaining the sites, teaching classes, taking classes, and planning for my prospectus. Sticking to the single install model would work good there.
Php 5.3 is something we want to move to, but we are holding off on until the summer. Our plan is to keep our existing site structure to an extent this year and in the Spring, while using the spring to test out new ideas. Then, over the summer we'll upgrade the server's php and redeploy sites with our newly configured and user-centered designs. We managed to upgrade mySQL to a much more recent build a few weeks ago, but there are so many sites running drupal 5 and earlier builds of drupal 6 (old site admin didn't keep them up to date apparently), that moving now would be suicidal for most of our content.
I will definitely look into the APC issue, since I"m not currently away if we've got it on the server or not. If it has the type of performance gains you mentioned, that would definitely be something we'd be interested in.
Thanks again,
V
I'm with Mike. The
I'm with Mike. The maintenance of that many sites on separate code is a management nightmare. I run about 70 sites on a single install. Many have SFTP access to their sites/*/modules and themes directories so they can add custom modules and themes. It's working smoothly and they get all the benefits of having their own install without making it a management nightmare for me.
Caching-wise, a single install is vastly more efficient, since all the Drupal core code and sites/all/* - which would be identical for all sites - would be cached. Breaking it up into multiple installs means that it has to cache multiple copies of the same code.
If you haven't already, check out Aegir (http://groups.drupal.org/hostmaster2). Between drush (an absolute must-have for anyone that runs more than a handful of sites) and Aegir, a lot of your management headaches go away. I use drush heavily and am in the process of rolling out Aegir on my infrastructure. Aegir will do a lot of the stuff you need - even to the point of providing the site owners the ability to back up, restore, clone their site (if you allow them).
Drush is on my todo list, but
Drush is on my todo list, but keeps getting pushed back a little bit every week. I really need to just sit down and work with it. As far as Aegir, we've looked into it, and I've also got it on my todo to set up on my localhost test server for investigation. The main barrier for using Aegir on our current setup is that we have Fully Managed Hosting with Storm on Demand, which doesn't allow a custom server tech like Aegir, and instead puts in WHM and C-Panel. What we've discussed doing though, however, is offloading our standardized sites for limited use instructors (they don't want to mess with it at all) onto a university server, which would allow us to implement Aegir. We have discussed the move from a fully managed to other models of support, but my boss is very happy with the 24/7 support fully managed gets him.
MT-Drush
I wish there was a multi-threaded version of drush so it could do 4 sites at a time... yes drush is a must.
MT-Drush
Names fail me at the moment, but there are plenty of OS-specific wrapper scripts that will "go to this directory and fork this other script in the background".
Perhaps this node.js script
It's based in the same asynch non-blocking model nginx uses. Not threads. Here's something that might apply:
Implementing simple work queues. From the article:
It uses a JS array for implementing the queue.
I concur
on the nginx suggestion. Yes it's less usual, hence more "difficult" to setup. But the configuration is much cleaner than Apache that it's quite a patchwork IMO. It's a question you must settle yourself with a cost/benefit analysis. Note that nginx has no .htaccess. You can replicate a few aspects of .htaccess in nginx or something along those lines, but there's no concept of a directory context, like in Apache. There's only, global, http, server and location contexts. Locations may map directly to the filesystem (static) or not (dynamic).
I would definitely avoid using PHP 5.3, since unless your sites use just a few unproblematic modules in what PHP 5.3 compatibility is concerned, you'll be treading in quicksand. Stick with 5.2.x. Wait for Drupal 7 before moving on to 5.3.x
I don't use php-fpm, since it's not available in Debian for 5.2.x and I don't want to incur the cost of creating my own debs. It's quite messy. You can use PHP FastCGI with Apache (mod_fcgid) or nginx, and the "standard" php-cgi with a supervising daemon like monit or supervisord.
Use drush. Use drush. Use drush. Otherwise you'll get cooked slowy in admin UI hell with all those sites.
Additional question
Hrm...a quick addition: are there any settings we'll need to configure with APC if we enable it? And, if there are, are there any good online resources for tuning that configuration?
You will need to check it to
You will need to check it to make sure it has enough memory. Your distribution's package should come with a script called apc.php that gives a friendly web interface with stats. Then you can adjust the maximum size with
apc.shm_size=128or however much memory you need. Other than memory size, defaults are usually ok.
Depending on your distro, you may have to increase the system-wide shared memory size as well. Darn Ubuntu and its 30MB default.
Thanks for the information.
Thanks for the information. We're currently trying 128mb, which is double the php memory settings we have set up overall. Its still getting a cache full count message, but its doing a ton of work, and getting a lot of hits and few misses once it gets rolling, and currently at its setting has no fragmentation.
Thanks for the information.
Thanks for the information. We're currently trying 128mb, which is double the php memory settings we have set up overall. Its still getting a cache full count message, but its doing a ton of work, and getting a lot of hits and few misses once it gets rolling, and currently at its setting has no fragmentation.
You can also limit it to
You can also limit it to certain directories. I have my code in a directory called "drupal" so I do:
apc.cache_by_default=0apc.filters="+drupal"
and then it doesn't cache things I use very rarely like phpmyadmin. But I'm on a pretty low memory VPS, so more concerned about conserving memory.
Thanks again, that may be
Thanks again, that may be very useful. We have a few sites, for instance, that don't get a lot of traffic, and that if they were put into the cache would be taking up space that our big use sites (sites for courses being taught) would use. I may need to investigate limiting the caching just to our big-use sites to help with that, since like I said we don't really have too much consistant traffic on the other sites, just sporadic, but I'm noticing those sites filling up the memory I have allocated.
re: Php 5.3 and nginx
Thanks for that advice. We'll definitely stay away from 5.3 then until we hit Drupal 7. Our goal is to migrate to 7 at the first major rollout of sites that we can, once its stable. On our timetable, that is looking like Fall 2011.
nginx sounds interesting, and I may have to peek into it as a long term project. I anticipate and hope to be working with Drupal for professional reasons through my current time at Purdue and into my future positions, so getting a better handle of other and perhaps better options would great.