Hi All,
I've been thinking about ways to build a really robust site spread across multiple cloud servers (common DB server) for redundancy and performance reasons.
I had an idea and I'm not sure if it's ridiculous or not:
1) create a multisite instance where all users, sessions and nodes are shared.
2) host each subsite on its own server
3) set up the subsites based on task, so that heavy processing can be done (many modules say) in one part of the site, and have less active modules/usage on another
4) use DNS or load balancer to direct traffic to the subsites based on task. So customised newsletters could all be generated on newsletters.site.com and all the video encoding could be done on encoder.site.com etc etc.
My thinking is that doing things this way allows you to run a minimal set of modules on the 'main' site, and also allows you to scale for each task individually, so if chat.site.com is taking an inordinate amount of resources, we can scale that server only without worrying about the rest.
Also if one function goes down, it doesn't take the whole site with it.
What do you guys think? Am I reinventing the wheel here or could this be a valuable approach?
BC
Comments
yes, but
In General it definitely makes sense to various islands in your application where each of these server-groups is responsible for a certain task.
Please take the following 2 comments in consideration:
1) Drupal is not always best of breed :)
Of course we all love Drupal, but not in all areas it will be the best choice. So for doing e.g video-transencoing, being a chat server or sending out newsletters you definitely will find systems or applications that use resources much more efficiently. Or e.g. using GPU or special hardware for the job.
2) Many tasks have spikes.
When you not sending out newsletters everyday you will see that the systems CPU will be pretty freezing most of the day. So maybe here another (or additional) approach might be interesting. Using the API you might be able to fire up a couple of cloud-instances just for the time where the service is needed.
But anyway be aware that Drupal applications could not be "sliced" in a way you proposed out of the box. There is a lot of work on the road to bring such a system alive.
Cheers,
Carsten
thinking along the same lines
I did a screencast about structuring my multi-sites in this manner for much of what you're talking about -- break up the requirements, spread them across a larger set of multi-sited drupal instances, and then there's more ability to shift load around. Assuming you use webservices to link together data as needed and have single sign on stuff down so that the user is on multiple domains automatically it is at the very least a valid potential approach. I say potentially because I'm doing this with restrictions currently not allowing cloud based architecture and I'm only in a few months of having it deployed to a modest user base (<500 authed users w/ small amount unauthed traffic on network).
https://drupal.psu.edu/content/dslm-multi-sites-steroids for an idea of a multi-site structure concept or http://vimeo.com/60084560 demo'ing things in a much ealier state w/ diagrams that describe some of what you're talking about
Ex Uno Plures
http://elmsln.org/
http://btopro.com/
http://drupal.psu.edu/
Wonderful
Thanks for the feedback everyone! I'll look into it some more over the next week or so and let you know if I find anything new/interesting.
Cheers,
BC