24 websites , 12 languages, in 12 countries. Can drupal handle this ?

Posted by leunis on January 19, 2011 at 3:10pm

Hi,

I am working on a new internet strategy for a client. This client is going to launch 24 new websites in 12 different countries (read 12 languages). This means two websites per country. I am unsure about the traffic as they will also boost their marketing activities. Based on current figures I expect 25.000 unique visitors per day in total (for all 24 websites together). And I guess this will grow with 5% per month for the coming years to come.
Each individual website will need its own frond-end (design) and content management.

I know that Drupal 6 is capable to run such a configuration, but I am unsure about performance of the core software.I want to solve the hardware site of the story with loadbalancing running 2 webservers and a separate DB server, just to be scalable

Can you please provide me with some advice on this case?

Thanks!

Comments

In short use Drupal

Posted by mikeytown2 on January 20, 2011 at 3:19am

Good article on Drupal scaling; with your proposed setup you should be fine.
http://2bits.com/drupal-performance/presentation-34-million-page-views-d...

Use drupal's built in multi site to handle the 24 domains.

I agree with mike.

Posted by threading_signals on January 20, 2011 at 11:58am

Multisite is the better way to go, as domain access has issues with a variety of i18n modules. There's two that I know of, which fixed will make it strongly worth considering, since it gives some centralized management options through the batch page. The 25k number is safely within range of what 2 webservers can do, assuming the proper amount of ram. The biggest technical challenge will probably come from CDN planning.

Mike, have you tried out domain access? I'm thinking about multi site use instead until they get the problems fixed up.

Domain Access

Posted by mikeytown2 on January 20, 2011 at 6:29pm

We use Domain Access with multi site to manage over 1k TV news websites across 50+ databases on 3 DB servers. So this is a platform that gets LOTS of traffic. We have a couple of patches for DA, but out the box it works great for us. We use varnish for anonymous traffic and we have boost in there with a custom java crawler. Long story short we set a 10 day cache lifetime for varnish and new content gets out the door in under a minute most of the time. It's a sweet setup when it works correctly (we are still debugging it, although it's mainly working correctly).

For the CDN we use FUSE S3 and mount the files dir on S3. If running with this setup, be sure to use this module http://drupal.org/project/imageinfo_cache. We use memcache as well, it's been a lifesaver for us; use the latest dev, a lot of bugs have been fixed in it now.

Domain + multi-site?

Posted by mshmsh5000 on January 21, 2011 at 9:46pm

mikeytown2, can you give a brief breakdown of the separation of labor between Domain Access and multi-site? I haven't considered using both before. Thanks in advance!

By Location

Posted by mikeytown2 on January 22, 2011 at 1:34am

http://www.komonews.com/communities
http://www.katu.com/communities
Every community in here is a domain listed on domain access
We use a different multi site install for each root domain.

It's a good match for what we do. Every use case is different.

On this site we use organic groups. Each city is a group; we thought about domain access but for this setup organic groups was a better fit.
http://galtime.com/

One DB, many options?

Posted by mshmsh5000 on January 28, 2011 at 3:46pm

Thanks for the examples. I'm just wondering what you use multi-site for when Domain Access is handling the domain-by-domain breakdown. They all have to be in the same DB for Domain Access to work, unless I'm missing something there; so what site-specific elements do you store in the multi-site dirs?

This is significant research for me right now because we're considering transferring thousands of tiny mini-sites to Drupal, and I'm considering the best architecture for this -- i.e., one massive multi-domain installation or something more federated.

Thanks again for the info.

Content Separation

Posted by mikeytown2 on January 28, 2011 at 7:19pm

The key to all of this is how separate you want your content to be. With domain access and organic groups, you can have content from one "location" appear on multiple or all locations by checking boxes on the node edit/save page. If you want all your sites to be separate from each other then I would go fro the multi-site only approach. The nice thing about Drupal is you can design it exactly how you want it to work. We give you the tools, it's your job to research and test which ones work best for building the site you need.

The other thing is performance; by having each one be it's own multi-site, thats a database that we can move to a different box. Right now we have 3 database boxes holding 50+ Drupal installs.

I haven't tried out aegir

Posted by threading_signals on January 28, 2011 at 9:55pm

I haven't tried out aegir yet, but I assume that it helps manage multi-sites (thus multiple databases), and your multi-sites can have domain access as Mikey noted. Since drupal's primary purpose is toward serving web pages, once traffic increases, load balancing can come into the picture, in which case replication and parallel queries are software options, and that would also mean multiple databases. I've been looking at pgpool, so it seemed relevant here. You asked a question that requires a technical perspective, then you realize that you asked a more business oriented question. For performance, having multiple databases is the answer, and for more functionality using multiple data pools, integration is the answer (views as far as I know cannot look into other databases, but I don't see why it can't be done), and one db is one common answer to solving that.

Clarifying

Posted by mshmsh5000 on January 29, 2011 at 2:26am

My question is purely technical. Here are the typical use cases for each approach:

Domain Access Single database, single codebase, federated content / users / features across multiple domains

Multi-site Single or multiple databases, single codebase, separate content, federated code across multiple domains

In a typical Domain Access-driven app, all my modules and themes are in sites/all and I have a single settings.php file. In a typical multi-site-driven app, I have common modules and themes in sites/all, and then can customize config, database settings per domain, and add modules or themes to each domain, by populating sites/example.com and sites/example2.com.

The cost of Domain Access is that you need a single database to track all domains and domain features. The cost of multi-site is that

My question, then, is in a hybrid situation where Domain Access is managing the different domains served, content access, users, config, themes, etc., what is there for multi-site to do? If I look in one of the multi-domain directories in a hybrid installation, what do I find?

Side note: Views can easily access multiple databases -- for instance, http://groups.drupal.org/node/27178

Mmm, sentences

Posted by mshmsh5000 on January 29, 2011 at 4:40am

Apparently I'm not the kind of guy who finishes sentences. I think I wanted to say something like, "The cost of multi-site is that domain-specific differences are all captured via file structure."

So I can see in the abstract how it could be great to hybridize the approaches, but I'm wondering how anyone has realized this in concrete terms.

Thanks for the link, that's

Posted by threading_signals on January 29, 2011 at 12:41pm

Thanks for the link, that's good to know of some of the steps taken. This is in the abstract, but I already touched upon the data redundancy and performance use-cases.

Provisioning and migration: beta test site vs production, drupal 6 to drupal 7, mysql to postgresql and vice versa; switching from one virtualization image to another, kernel upgrades, etc. should be possible.

Database/distribution/platform/organization specific features: PostGIS, Pressflow, use of tomcat/apache solr module integration vs not using it, use of nginx etc., openatrium...?

Security: permissions levels, policy levels, setups.

Cost/Resource Utilization: CDN, APC vs Memcache usage, etc.

It depends on the target audience needs.

Mikey is the best..! ;) That tip on the hybrid approach was very useful. Anyway, it will depend on your use-case, so you would have to customize and research as needed as noted above.

We are working with a client

Posted by agoel@axelerant.com on January 20, 2011 at 1:14pm

We are working with a client currently helping them with a big mess on a multi-site installation using domain access. At your required traffic level there should not be many issues, but as traffic continues to increase and more editors need to login to the back-end, the database will begin to become the bottleneck if the app is not well designed.

Just something to consider for the long-term.

You could also do it

Posted by perusio on January 21, 2011 at 8:55pm

at the server layer using something like GeoIP and thus avoid touching the PHP/Drupal layer.

You deploy one backend for each locale. They can be running in different machines or a single machine in diverse ports. Of course if you plane to use the stuff about content translation management and such you need to set up a scheme for managing the content across all backends.

Just and idea.