Determining the ideal server size to handle 4M page views per month

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Agiss's picture

What would be the ideal server size in terms of CPU and memory to handle around 500K visitors per month with an average of 8 page views per visitor. This would generate 4M page views per month.

The website is a dynamic classified ads website built on Drupal 7.54, PHP 5.4.44, MySQL 5.5.42.

In addition to this the system will handle the creation of the classified ads submited by authenticated users.

An average page will have 1.6MB in size and 180 requests.

Thanks

Comments

The most important to know

gor's picture

The most important to know next Params:
- peak of page requests/per second
- MySQL database peak bandwidth/per second
- max sql query time
- do you use views and are they cached?

Also php7 with opcache, filecache module or memcache will increase performance
ssd or nvme drives, hardware raid are must have too.

If you need managed solution, contact me.

I have worked on a site that

kevinkhuat's picture

I have worked on a site that had a beefy server with 64 gb of RAM, and the site was still performing poorly. If the correct caching strategy is not in place, I don't think it would matter how strong the box is.

Other than the authenticated users who post classified ads, it seems like most of your traffic are from anonymous users. You can use a CDN like Cloudflare to absorb the bulk of the anonymous traffic.

For dealing with authenticated user traffic, I had success added a slave db to handle most of the db reads.

Here's a link with some more information.
http://redcrackle.com/blog/how-use-drupal-7-mysql-master-slave-setup

Thank you guys,Kevin, having

Agiss's picture

Thank you guys,

Kevin, thank you for sharing this. Having an external DB is a good option but, it will double the hosting cost. Nevertheless, we are actually looking into this alternative.

Gor, the website is not live as yet and unfortunately I don't have any peak loads information. However, I am expecting the rush hour to be between 7pm - 11pm.

Yes, I am using OPCACHE and Views which are cached for non authenticated users, will also have SSDs and RAID1

I am trying to find a fair cost/power balance and completely baffled about CPU and memory. A Quad core Xeon E3-1230 3.30GHz costs 2 times a Quad core Xeon 2.4GHz.

BTW, I am refering to dedicated servers.

@agiss DO not try to save on

gor's picture

@agiss DO not try to save on CPU. Also do not buy 2.x Ghz series with multiple cores.

You do need as fast as possible CPU (it has positive side-effect on Drupal sites)

Having external DB is good but it will affect query performance for sure.
Specially with VIEWS. VIEWS are hungry for MySQL Bandwidth.

Regular server under my team management has avg 25mb/sec in peak time and it is not a full load for the server.

With such load, there is up to 20Mb/sec MySQL bandwidth.

If you have external MySQL even in a local network, it will generate huge bandwidth, so you will need fast switch and ethernet card for 1G (not 100M).

Also do not buy SOFT RAID - they are using CPU for doing RAID job and on power loss, there is high risk on DATA Failure.

I don't think external DB

Hadi Farnoud's picture

I don't think external DB query time is much of a difference. it's milliseconds slower

PS. check demo site:

gor's picture

PS. check demo site: http://demo.malleus.vps-private.net

There is debug info at the bottom:
Page execution time was 92.04 ms. Executed 135 queries in 15.44 milliseconds.

It's working on the server where multiple sites generated up to 25Mb/sec in peak time.
Peak time happen 14 hours ago. Right now it's only 5Mb/sec

Hi! I think Gor is right: You

dr.admin's picture

Hi!

I think Gor is right: You need as fast as possible CPU

I recommend you to start from one dedicated server with Xeon CPU ( 6-8 cores with multithreading ) and database on SSD disks.

I do 10M pageviews per month

kandrupaler's picture

I do 10M pageviews per month @ max 3k concurrent users on a digital ocean $40 / month droplet which I've optimized quite a bit.

you cached hell of a lot

Hadi Farnoud's picture

you cached hell of a lot then, no? are you using Memcache? how often site is updated?

Yes, 1GB for Memcache.

kandrupaler's picture

Yes, 1GB for Memcache. Digitalocean's SSD also helps a lot.

A bit old post,but what are

pinkonomy's picture

A bit old post,but what are you doing to achieve so?Are you using a Reverse Proxy?
What are the characteristics of the droplet?
Thanks in advance

mikeytown2's picture

Long story short, just about anything modern should be able to handle that workload.

Update to PHP 7.1. That will give you roughly a 4x improvement over PHP 5.4.

MySQL is the bottleneck for Drupal. Use MySQL 5.7 (or most recent Maria DB/Percona), you should see some big improvements in terms of concurrency vs 5.5. Ram is important for MySQL, knowing how big your database is in GB would be nice to know but with ram being so cheap might as well load it up pretty good; start at 32GB and go from there. Put the DB on a SSD (SM863 or better) as well for faster writes.

The last thing to consider is the cache layer. This is usually what causes Drupal to melt. Core's is junk under heavy load. Memcache/Redis is always a good option, that's just one more thing that you'll have to manage though. You can use MySQL as the cache layer if you use it differently; there are 2 modules that do this well: https://www.drupal.org/project/lcache and https://www.drupal.org/project/apdqc. I'm using apdqc and for us it just works (granted I wrote it so it better work). I'm hoping to use the lcache db schema in the future and try to merge the 2 together as async writes are really nice.

Now that your server isn't going to melt down I'd focus heavily on the frontend. The frontend is where some huge performance gains can be found. For this http://drupal.org/project/advagg/ is my recommendation. I like to test with the network at 2G edge speeds. Take https://www.webpagetest.org/result/170420_DP_1G91/ as a quick example from Gor; start render is 7.8s. Here's an example from a heavy site https://www.webpagetest.org/result/170420_CK_1GA5/ start render is 5.7s even though it's downloading 720kb vs 88kb. That's the magic of optimizing for the frontend.

Thanks a lot for your input

Agiss's picture

Thanks a lot for your input guys.

We are going to start with a dedicated server, an Intel E3-1230v5 (3.4 GHz, 4 Core) with 64GB RAM and SSDs on RAID1 also, we'll use opcache and advagg.

I will be posting updates

Cheers

I think you're overdoing

kandrupaler's picture

I think you're overdoing overdoing :-) Pls see my other comments.