scalability

I am working on a web application which will primarily be used by authenticated users. The web application imports records from CSV files using the Feeds module, MongoDB was chosen to store this data because it is much faster to do the writes rather than in MySQL. The application is importing thousands of records every day really fast. The records are referenced to the users and when they log in, they can consult their information. The application is also used as an API for the mobile app.

I have a 17gb AWS EC2 Instance as the server.

A brief summary of the idea:

Data within entities that do not need to be processed individually within a view (or better: are not required to be addressed individually in any case) should be stored in a single database table, rather than being spread over N tables like they would if stored by the Field API "the usual way", in order to reduce the number of (sub)queries or subsequent queries and hereby reduce the load on the site's database.

Last Wednesday I gave a presentation at OxDUG on how we did a recent site build of Love Food Hate Waste (LFHW) and its subsidiary sites.

Please find the slides attached. I hope they're of some use!

LFHW was build by Torchbox, my employers. The overview of the site was as follows:

<

ul>

Hi,

I'm an architect working on a project to rebuild our current consumer site, possibly using Drupal, and am new to Drupal and learning as much as I can as fast as I can. We currently serve all users to our site out of a single colocation (colo), but in the near future are planning to add additional colos in other geographic locations and serve the site concurrently from them all, with users being routed to the nearest colo.

Hello,

I'm looking for a forum solution for a Drupal site. This forum must support moderating. Is there any lightweight solution, or is the forum access module scalable? Last year I had problem with a node access module. Is it correct that Forum Access sustains the same issue?

Do you know any big forum (a few dozens of thousands of nodes or more) that uses the Drupal forum?

Thanks.

Hi folks,

A client of ours just migrated their Drupal site to a shared hosting service and now experiencing performance issues with logged in users. Drupal's caching and APC are enabled, so content is loading fine for anonymous users. We are mostly running into issues with node add forms and administrative pages loading very slowly.

Has anyone run into similar issues migrating a site from a dedicated server to shared hosting? How did you improve performance on your site?

I have a general question about stress testing a site. How do you guys do it? I am building a site under less-than optimal conditions and I want to know what the fail point will be.

Hi Folks,

It was great seeing everyone last night. I hope you all had a good time.

I wanted to pick your brains about a topic I know we all love - scaling Drupal!

Looks like I generated a little buzz when I mentioned in my SxSw talk that I'd recently done a proof of concept scaling Open Atrium to 200+ requests/second using EC2. Since a number of people have asked about it, here's the skinny:

Disclaimer

First of all, I need to point out that this was a proof of concept build designed to let me know if the use-case I was pursuing (1000s of logged in users in a "burst" scenario) was even feasible or not. I cannot guarantee these results for your business case; your mileage may vary, etc.

The Basic Setup

I'm in the process of setting up Drupal as the CMS for a large community site. This is is going to get a fair amount of image/video uploads which are put into /sites/default/files/ This works well for a few hundred to a few thousand files but after that performance and maintainability starts to degrade.