The basic question is how to optimize performance for authenticated
users on site with heavy forum traffic that requires membership. In
general performance is adequate but there are spikes of heavy traffic.
Site is for a band. On occasion the band member
will tweet and link to the site and hundreds of users will log in to
see what is going on. Thus the traffic spike.
Metrics: Highest traffic in the last 60 days was 3667 visitors in a
24 hour period. Of this 3317 users were authenticated users with
accounts. This is according to Google Analytics.
Site has a lot of modules views, panels, cck, advanced forum, advanced
profile, author pane, and several more etc ...
Here is what we have done so far:
Hosting environment
6 headed web server with 2 headed db server
APC is installed
Varnish is installed and working wonders for anon traffic but that is
only 10% or so of site traffic.
Memcache is installed and I see it working as cached pages have
significantly lower number of queries after they are cached.
Mysql query cache is working well as queries that are rerun are very
fast like 0.01 sec
Still php exec load time is over 1.3 sec for a forum node and 900ms for /forum
Additionally hitting /forum with apache ab (and a session with -C
SESSIONID) it seems we can only handle about 10 requests per second
- but not sure if that would be per server (think so but not sure).\
When profiling the site with xhprof on a local machine it seems that the biggest problem is that there is a lot of code to run 144,000 functions to generate a forum post node with comments. Worst offender was cufa function being called by author pane module yet disabling the module only showed a 10% or so improvement in php execution time.
We did find a few really slow queries and adding a couple indexes on the comments tables really helped.
Would love suggestions for being able to handle these traffic spikes.
Thank you
Comments
web server
Hi,
You did not mention what web server you are using, version of php and the OS? Is this Apache, nginx / php-fpm, lighttpd or some other rig?
At the mysql level besides creating indexes, look at queries that use subqueries and possibly can be converted to joins,
ORDER BY optimizations and so on. There is also quite a bit of tunning that can be done at APC, memcache and mysql. These all may
impact what's being done on the backend.
Do you use authcache?
Do you use authcache?
-Natalie
Friendlydrupal.com - Drupal Video Tutorials
Tuning
Database:
Sounds like you have done a lot already. This tool might allow you to gain more insight into the database layer of your stack. http://code.google.com/p/maatkit/
Also give the latest dev of DB Tuner a try, it will help you identity slow view queries.
MariaDB with XtraDB instead of InnoDB is always a good choice. If your looking for very bleeding edge try out the facebook patch for MySQL: https://launchpad.net/mysqlatfacebook
Pressflow:
Use it - http://pressflow.org/
Servers:
When they are under a heavy load (push them almost to the edge) what is the bottleneck? CPU?
apc - what does apc.php tell you?
memcache - you can place almost every cache_* table into memcache; be sure to test it though.
If your still stuck, give panels caching or authcache a try. Be sure to ID what your bound by (CPU, MySQL, DiskIO, etc) before making major changes.
You need to use Authcache
Hi
Your website has a lot of authenticated traffic. So you need cache for authenticated users.
So, try to use authcache module (Increases peformance to about 5000% to 20,000% for registered users - at least for me).
Try to load user-specific blocks via ajaxify_regions module to avoid there caching without affecting the caching of other page elements on the same page.
Hope this helps
DIMSKK
Check Authcache + Easy Authcache
Guys suggest you to use Authcache, and they are right.
From my side I suggest you to use also Easy Authcache http://drupal.org/project/easy_authcache , that will makes it's implementation faster