Posted by v1p3r1c3 on May 31, 2010 at 1:37pm
Hello,
What server spec's do i need to host my high traffic site.?
I have an existing dedicated server with specs:
• Fully manage
• Intel Dual Xeon E5504 Quadcore
• 4GB DDR3 SDRAM
• 1 SATA DRIVE
• 1 SATA Backupdrive
• CentOS 5 64bit
• 8000Gb Monthly Transfer (4000 in + 4000 out)
Drupal:
Drupal 5
90+ nodes
400MB database size
20 authenticated users
Mostly Anonymous
Caching:
Boost
CSS
Gzip
Block Cache
Traffic:
2k+ visits/hr
20million page views+/Month
Comments
It's really hard to tell, but
It's really hard to tell, but I would just throw APC (for opcode caching) on the server and it should be able to handle it with no problem. I run a site with 2 similar servers (one web + web db), boost, 40,000+ nodes, 1.5 million+ comments and an average of 200 logged in users at a time. A lot of days we see 20,000+ page views per hour and the server load on both machines never goes about 0.5.
With such a small database set, you could also use cacherouter and use APC to replace the native Drupal database caching. That would free up some space needed by MySQL for things like the query cache and keep some load off the database. Even if it looks like you are going to run low on memory, adding a couple extra gigs doesn't cost that much.
The one thing that could throw a wrench in the works is any modules or views that do some nasty queries. Just take a couple of hours with the devel module and look for any queries that take a long time to execute and optimize them (add indexes, rewrite queries, cache results, etc.).
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.
Dedicated db server
You mean you have a dedicated database server?
Yeah we do, but we are
Yeah we do, but we are putting out sometimes 10x the average pageviews you are expecting. Our database is also pushing 1.5 gigs right now and we get an average of 1,500 comments per day plus 40 nodes. We also have over 30,000 users. I actually ran the site on one server right before the 2008 elections (it's a political site) because our database server was down. One day during that time we had over 60,000 page views in 1 hour and the site never went down. At the time we were just running memcache with page fast cache (not boost).
My suggestion is to get the site setup on the server and then load test it with JMeter. JMeter is very advanced and can take some work to figure out, but you can test everything from anonymous views to nodes being created to people crawling the site and posting comments. Here's a good article explaining how to do that:
http://www.johnandcailin.com/blog/john/load-test-your-drupal-application...
Basically you just setup your test plan and run it while monitoring the server in Top to see what your loads are hitting at. That way you can get an idea where your bottlenecks might start occurring and we can figure out how to squeeze extra performance from that area of the server.
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.
We tried dedicated db server
We tried dedicated db server with no private switch, our server sends almost 40m bits per second based on our bandwidth monitoring which cause high bandwidth consumption. It shows that our drupal sends lots of queries to db server even we already tune these settings read_buffer, sort_buffer, join_buffer, table_cache, thread_cache, query_cache_size, key_buffer_size.
We clone our drupal on our development site and used siege with 300 concurrent users. It was a good test results but when we put it on our production site, still we experience downtime of our web server. Every time we put a content it will increase memory consumption and bandwidth going to db server.
Do you think we have a bad implementation of our views or most likely drupal implementation.?
As of now our site can handle 2k to 2.5 visits /hr but if it reach 3k it will slow our server and site access. The bigger problem we have right now is when we add a content it will take 10 or 30 seconds to fully submit the content.
what would be the best and fast solution for us as of now? Upgrade our Server or we implement those performance and optimization strategy?
There very well could be a
There very well could be a view that is messing with things. When you post a new node, the query cache for those tables will be invalidated, so regenerating the query might be taking up some time. Enabling the slow query log in MySQL would help find those. Set the long query time to 1 second, since you really never want any queries longer than that.
Also if you are running anything like a bunch of node access or trackers, you could be seeing long times from MySQL waiting for locks. Check your MySQL logs and see if there are a lot of table_locks_waited in there. If there are, then switching those tables over to InnoDB would help, but you have to watch your memory also.
For a big help, I would set up APC so you get some opcode caching (that will save on hard drive reads and compiling on every page view). Check to see how much free ram you have after that under normal load. If you have enough (say 250 mb or so), then install the cacherouter module and use APC's user cache to replace all your regular database Drupal caching. That will take quiet a bit of load off the database and improve the load times of Drupal. With APC alone as an opcode cache, you can see an average of about a 25% performance boost. Even if you are running low on memory, I would just look into having a couple of extra gb put into the server so you can move the object cache to memory. Generally adding a couple of gigs is only a few dollars extra a month with hosting companies.
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.
our hosting use eAccelerator
our hosting use eAccelerator for PHP accelerator & optimizer
"It shows that our drupal
"It shows that our drupal sends lots of queries to db server even we already tune these settings read_buffer, sort_buffer, join_buffer, table_cache, thread_cache, query_cache_size, key_buffer_size."
It's great that you are tuning your DB server, but that's probably not going to significantly change the database's portion of page generation time. If your problem is ping time between the DB and web servers then the only thing that you can do (beyond physically reducing the barriers to that ping time), is to reduce the number of queries per page. Caching is the easiest way to do this. And, depending on what tools you've used to build your site, the block cache can go a long way here.
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
Installed memcache
I installed memcache and the status says:
curr_connections 23
total_connections 24
connection_structures 24
cmd_get 4215
cmd_set 334
cmd_flush 8
get_hits 3070
get_misses 1145
I'm not familiar with memcache how do I do manage it.