My web is still slow even with Boost + Authcache + APC + Memcache

Posted by reynaldio on April 13, 2010 at 10:55am

Hi all,

i recently installed boost, authcache, apc and memcache. Is this too much? However i dont get the performance boost i want. My web even getting slower than ever. could there be conflict between this modules? With boost and authcache alone, page execution between 2800-3500 ms. Later i installed apc and page execution goes beyod 30000-50000 ms. And then i installed memcache and page execution goes back to 3000.
What have i done wrong.

php.ini

max_execution_time = 60
memory_limit = 320M

extension=pdo.so
extension=pdo_sqlite.so
extension=sqlite.so
extension=pdo_mysql.so
extension="ixed.5.2.lin"
;zend_extension="/usr/local/Zend/lib/Optimizer-3.3.9/php-5.2.x/ZendOptimizer.so"
extension=uploadprogress.so
extension=apc.so
extension=memcache.so

apc.enabled=1
apc.shm_segments=1
apc.shm_size=256
apc.apc.stat = 0
apc.ttl=0
apc.include_once_override = 1

memcache.hash_strategy="consistent"

settings.php

$conf['memcache_servers'] = array('localhost:11211' => 'default');
$conf['cache_inc'] = './sites/all/modules/authcache/authcache.inc';

any help and idea will be highly appreciated

Regards,
Reynaldi

Comments

When doing performance tuning

Posted by dalin on April 13, 2010 at 11:14am

When doing performance tuning you can't just install a bunch of things that might help and hope for the best. You need to first figure out what your bottlenecks are, benchmark, make one performance improvement that fixes that bottleneck, benchmark to confirm that you actually made an improvement, repeat. It's so easy to make a small error in configuration that will actually make problems worse. If you don't know what the problem is, and you don't benchmark, then you can't fix the problem, or even know if you've made things better or worse.

Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his

Benchmark tools

Posted by reynaldio on April 13, 2010 at 11:18am

Hi,

Thanks for you input
what tools do you use to benchmark your site performance?

Regards,
Reynaldi

Apache Bench (ab) is probably

Posted by dalin on April 13, 2010 at 11:24am

Apache Bench (ab) is probably the easiest tool that you have available.

In reviewing your config above I see that you have your PHP memory limit at 320MB. If you really do need it to be that high then you have a serious problem that performance modules aren't going to be able to fix. A standard Drupal page usually uses somewhere in the range of 24-48MB. Some admin pages may require a bit more. If you need 300MB to render a page you likely have a misbehaving module. Likely a custom module, a custom theme, or a lesser-used module.

Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his

Hi, i put 320M memory_limit

Posted by reynaldio on April 13, 2010 at 12:07pm

Hi,

i put 320M memory_limit is only for precaution sake. It usually only takes 80-90mb.
does setting memory_limit high have negative impact?

Performance is very good when using boost only. It only takes 5 secs to open the homepage. But boost is only for anonymous guest only. I need improvements for registered users. That is why i installed those other modules. Do those modules work well together?

Regards,
Reynaldi

and 1 more thing, i get B on

Posted by reynaldio on April 13, 2010 at 12:20pm

and 1 more thing, i get B on YSlow. But still that does not reflect page loading performance on my web :(

Memory?

Posted by yohgaki on April 13, 2010 at 1:08pm

Generally speaking, APC will never slow down execution. Do you have enough memory? I guess you don't.

大垣靖男 / Yasuo OHGAKI / yohgaki@ohgaki.net
http://www.ohgaki.net/ / http://www.es-i.jp//http://www.provephp.com/

Hi, My server is running on

Posted by reynaldio on April 13, 2010 at 2:06pm

Hi,

My server is running on Hostgator VPS account with 756MB of memory. I've checked and i still have 512 Mb of free memory.

Regards,
Reynaldi

How it is possible?

Posted by jcisio on April 13, 2010 at 2:29pm

768 MB total, 80 MB for each PHP process, a few dozens of MB for OS, and you have more than 500 MB free?

Did you check if your APC setting is correct (in phpinfo, e.g. APC memory usage), 256 MB is too much, too.

YSlow tells you only the frontend, it can't barely distinguish between 200 ms and 2000 ms execution time on the server side. As per dalin's suggest, do more benchmark. If a script takes 3000 ms to execute, how much for PHP and how much for MySQL? If PHP, tune your APC. If MySQL, look at your slow query log. And sometimes, improve can be only made with hardware upgrade. How many pageviews per day do you have? How many modules do you install? It's strange that a normal Drupal script uses 80 MB if it is not a large image manipulation.

Hi, That's what my WHM report

Posted by reynaldio on April 13, 2010 at 3:21pm

Hi,

That's what my WHM report says. I do have a lot image manipulation on my web.

         total       used       free     shared    buffers     cached

Mem: 786432 273792 512640 0 0 0
-/+ buffers/cache: 273792 512640
Swap: 0 0 0
Total: 786432 273792 512640

I have about 500-1000 page views per day and about 50 modules enabled.
I'll try to do more benchmarking and post back with the results.
btw, my website is www.warawiri.com

Regards,
Reynaldi

Database

Posted by stompersly on April 20, 2010 at 9:59pm

If your site is slow for authenticated users I would suspect you need to look at the database. Try tuning the MySQL as shown here: http://web3us.com/drupal6/how-create-web-site-handbook/how-setup-mysql/t...

Is your PHP running as CGI?

Posted by Alexander Langer (not verified) on April 21, 2010 at 10:14pm

Is your PHP running as CGI? In that case the APC cache may not be shared by different CGI processes and you won't see any speed improvements. Checkout this discussion which also covers workarounds: http://stackoverflow.com/questions/598444/how-to-share-apc-cache-between...

On the otehr hand it may be easier to switch to mod_php5 instead of running PHP as CGI.

yes!

Posted by playfulwolf on April 21, 2010 at 11:24pm

Agree - it can be the cause of problem.

^{drupal+me: jeweler portfolio}

disable memcache!

Posted by playfulwolf on April 21, 2010 at 11:21pm

Hi,

I had experience with similar setup on similar vps, and... almost the same problems. Try to:
1) Remove memcache from server and all related Drupal modules. And yes, I am sure, because it sometimes freezes when used with authcache and it is useless if you are using APC and Boost in your configuration.
2) Increase APC memory, to the amount of APC + memcache, maybe a bit less, there is apc monitoring script, so it will help a lot. WOW! In your case you already allocated 256M - it is an overkil! Make 128M and give more/less after initial testing
3) install cacherouter module, and tune it to use 'apc' as storage (you allocated extra memory, right?)
4) install boost, enable html, maybe ajax and json caching and NOT css, js as you probably are using css, js aggregation modules - they create real mess if enabled both... I also do not recommend to enable Boost crawler...
5) Disable Drupal native core caching as now authcache will take care of that
6) In authcache settings enable cacheing ONLY for registered users (Boost will take care of rest)
7) Play with views caching and block caching settings, avoid any on-the-fly calculations, maybe with js, css agregation settings - but not with the most aggresive settings.

exact example: http://hotweb.lt

NOTE: I am not a great specialist of Apache settings - but you need your config be tuned for APC to function properly.
NOTE2: I take for granted, that you already set up VPS, MySQL, PHP and other non-Drupal stuff in the best possible way.
NOTE3: YSlow may help you with the front end performance, which is no less important
NOTE4: I suspect you are using settings for your VPS, as if it is quad-core machine with 8gigs of RAM... Probably the VPS you are using is in "low-memory server" category , so google for optimal settings.

Khalid from 2bits wrote some good articles, google them.

^{drupal+me: jeweler portfolio}

css/js caching in boost

Posted by mikeytown2 on May 28, 2010 at 5:02pm

I created the caching of .js and .css files just for use with the aggregation option. Otherwise you will get missing css & js files unless you set this boost setting that kills the boost cache on every flush cache request, since drupal_flush_all_caches clears out all aggregated css and js files. Whats left in the boost cache are html referencing css and js files that no longer exist.

Ignore cache flushing: Disabled
By default its set to Only Ignore Clear Entire Cache Commands (Recommended if caching css/js files)

Performance tuning

Posted by Mixologic on April 26, 2010 at 5:42pm

With performance tuning, I've found that there is almost no silver bullets. That being said you definitely need to know exactly what your setup is doing before tweaking anything.

If you want to get serious about performance tuning, isolation of bottlenecks and benchmarking, then the best tool available (imho) is xhprof.

This is the tool facebook built to analyze and monitor their systems, and lucky for us they open sourced it last march.

Its a hierarchical code profiler that gives you CPU and Wall clock times for how long it takes to run through your code, and comes with a web based tool to parse and digest all the output from the profiler, making it really, really, obvious where all the holdup is. (be it mysql, or particular modules etc).

For help installing it and more details I highly recommend this article:
http://techportal.ibuildings.com/2009/12/01/profiling-with-xhprof/

Btw, there is a misleading comment in the comments of that ibuildings article for y'all wanting to install xhprof on your mac: comment 8 says just use macports to install it.. you have to be careful if you are running snow leopard, and make sure that it doesnt try to compile it as 64 bit only .so, when all your php could be compiled 32 bit (like if you are using mamp).

Oh one more thing, when you downloaded and installed APC, there should have been and 'apc.php' file in the directory with everything else. Throw that somewhere accessable on your server (like www.warawiri.com/apc.php), and give us a screen shot of your output. That'll also display all the other apc settings, which may or may not need adjusting. Very useful tool for troubleshooting the apc cache.. (also, edit the apc.php file and change the password in the file.. that will allow you to 'login', which will allow you to do stuff like flush the cache etc)

Hi, sorry for slow

Posted by reynaldio on April 27, 2010 at 2:51pm

Hi,

sorry for slow response.
@PlayfulWolf so, memcache is not compatible with authcache?

i checked with webwait.com, and now my web loads in 3 secs for guest and 10 secs for auth users which is a lot of improvements (50 secs before).

but my page execution time needs improvements (devel: 2607.18 ms)

I've put apc.php in my drupal directory. you can view my apc in http://bandung.warawiri.com/apc.php
i see that apc uptime is always 0, which means that it is not working. i have no idea what could have been wrong in my config.
should i enable fast cgi?

this is my apc config in php.ini

apc.enabled=1
apc.shm_segments=1
apc.optimization=0
apc.shm_size=128
apc.stat=0
apc.ttl=0
apc.user_ttl=0
apc.num_files_hint=1024
apc.mmap_file_mask=/tmp/apc.XXXXXX
apc.enable_cli=1
apc.rfc1867=0

Thanks
Reynaldi

You're site isn't using APC

Posted by jaydubb181 on April 27, 2010 at 5:03pm

reynaldio , I looked at your APC cache graph, and you're site isn't using APC much at all. You've only got 1 entry in the APC cache and it's your apc.php file. So either memcache or authcache is handling your caching here.

That's what i'm trying to

Posted by reynaldio on April 27, 2010 at 5:28pm

That's what i'm trying to identify. i still havent figure out what i've done wrong. maybe you can point out something that could possibly the problem?

Thanks
Reynaldi

APC definitely not working.

Posted by Mixologic on April 27, 2010 at 11:29pm

Thanks for putting your apc.php up.. This helps a lot. apc isnt being used at all.

What it tells me is that your httpd daemon is not staying alive between requests. This is probably happening due to the way you have apache configured, if its running in cgi mode, or as suPHP, its basically firing up a new process, parsing&compiling the php, delivering the content and then terminating.

two things you can do to dig deeper:
a. create a phpinfo.php file in the document root with the following in it:

<?php
phpinfo();
?>

That will spew out everything about how php is running, including a lot of apache config information. if you see suPHP in there, thats probably the issue.. you'll need to switch it to run mod_php5 or mod_fastcgi. It depends on how your server is configured of course, and this can lead to some permissions issues when switching from suPHP to mod_php or mod_fastcgi, as apache will probably run as "nobody" instead of whatever user it used to run as.

It might be that apache is configured with mod_php5 or fast_cgi, yet configured to tear down immediately as well.. if thats the case you'll probably have to post your /usr/local/apache/conf/httpd.conf file for more clues.

APC

Posted by Mixologic on April 27, 2010 at 11:48pm

One more note about apc, even with memcache or authcache on, apc should still do opcode caching.

There are two types of caching going on, the first is opcode caching. When the php file is compiled into opcodes, APC, xcache, and eAccelerator all take those compiled opcodes and save em in memory. The next time apache goes to compile the php files, it first checks the cache to see if they are already done (but the process still has to be alive from request to request).

The second type of caching is object (or key/value store) caching. Object caches let you store anything.. database query results, variables, entire html pages - whatever you want. APC,xcache, and eaccelerator are all object caches in addition to being opcode caches. Memcached is also an object cache, but not an opcode cache.

The main difference between memcached and apc/xcache/eaccelerator is that memcached is designed to be distributed, so that if you have 5 webservers all accessing the same data, and the same data should be cached, they can all share the same caching resource. If you only have one server however, you might as well simply use the object caching capabilities of your opcode accelerator, as memcached is just one more thing to configure and tune.

Read this link for more info

Hi, sorry for my slow

Posted by reynaldio on May 2, 2010 at 4:47pm

Hi,

sorry for my slow response.

@Mixologic Thank you for your detail explanation. i do not see suPHP in phpinfo(). I've been trying to configure my hostgator account to use mod_fastcgi but to no avail. I've asked hostgator but they said they cannot help me with this and suggested me to post my question to drupal forum :(

Do you have any articles or links i can follow? I have full access to apache.conf and php.ini.
Please also note that i use domain.module, and hostgator said that this is the issue why i cant get fastcgi to work.

I put "AddHandler fcgid-script .php" in my .htaccess as hostgator suggested but still not working. Instead it pop up download page to download index.php.

Thanks in advance
Reynaldi

Below is my httpd.conf, in

Posted by reynaldio on May 2, 2010 at 6:16pm

Below is my httpd.conf, in case you need it. i do see suPHP in my conf but have no idea how to change it without breaking apache :(

Thanks

httpd.conf

Posted by Mixologic on May 3, 2010 at 3:40pm

Cant see your httpd.conf.. sometimes attachments dont work.. try again?

Also, if you want to pm/dm me a link to your phpinfo, that'd be useful too.

Hi, sorry, this is my

Posted by reynaldio on May 6, 2010 at 6:09am

Hi,

sorry, this is my httpd.conf

Include "/usr/local/apache/conf/includes/pre_main_global.conf"
Include "/usr/local/apache/conf/includes/pre_main_2.conf"

LoadModule frontpage_module modules/mod_frontpage.so
LoadModule bwlimited_module modules/mod_bwlimited.so
LoadModule auth_passthrough_module modules/mod_auth_passthrough.so

Include "/usr/local/apache/conf/php.conf"
Include "/usr/local/apache/conf/modsec2.conf"
Include "/usr/local/apache/conf/includes/errordocument.conf"

ErrorLog "logs/error_log"
DefaultType text/plain
ScriptAliasMatch ^/?controlpanel/?$ /usr/local/cpanel/cgi-sys/redirect.cgi
ScriptAliasMatch ^/?cpanel/?$ /usr/local/cpanel/cgi-sys/redirect.cgi
ScriptAliasMatch ^/?kpanel/?$ /usr/local/cpanel/cgi-sys/redirect.cgi
ScriptAliasMatch ^/?securecontrolpanel/?$ /usr/local/cpanel/cgi-sys/sredirect.cgi
ScriptAliasMatch ^/?securecpanel/?$ /usr/local/cpanel/cgi-sys/sredirect.cgi
ScriptAliasMatch ^/?securewhm/?$ /usr/local/cpanel/cgi-sys/swhmredirect.cgi
ScriptAliasMatch ^/?webmail/?$ /usr/local/cpanel/cgi-sys/wredirect.cgi
ScriptAliasMatch ^/?whm/?$ /usr/local/cpanel/cgi-sys/whmredirect.cgi

Alias /akopia /usr/local/cpanel/3rdparty/interchange/share/akopia/
Alias /bandwidth /usr/local/bandmin/htdocs/
Alias /img-sys /usr/local/cpanel/img-sys/
Alias /interchange /usr/local/cpanel/3rdparty/interchange/share/interchange/
Alias /interchange-5 /usr/local/cpanel/3rdparty/interchange/share/interchange-5/
Alias /java-sys /usr/local/cpanel/java-sys/
Alias /mailman/archives /usr/local/cpanel/3rdparty/mailman/archives/public/
Alias /pipermail /usr/local/cpanel/3rdparty/mailman/archives/public/
Alias /sys_cpanel /usr/local/cpanel/sys_cpanel/

ScriptAlias /cgi-sys /usr/local/cpanel/cgi-sys/
ScriptAlias /mailman /usr/local/cpanel/3rdparty/mailman/cgi-bin/
ScriptAlias /scgi-bin /usr/local/cpanel/cgi-sys/scgiwrap

Options All
AllowOverride All

Options Includes Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all

Order allow,deny
Deny from all
Satisfy All

Order allow,deny
Deny from all

Satisfy All

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common

CustomLog logs/access_log common

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio

ScriptAlias /cgi-bin/ "/usr/local/apache/cgi-bin/"

AllowOverride None
Options None
Order allow,deny
Allow from all

TypesConfig conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz

PidFile logs/httpd.pid
LockFile logs/accept.lock
# Defined in /var/cpanel/cpanel.config: apache_port
Listen 0.0.0.0:80
Timeout 300
User nobody
Group nobody
ExtendedStatus On
ServerAdmin email(at)warawiri
ServerName warawiri(dot)com
LogLevel warn

# These can be set in WHM under 'Apache Global Configuration'

ServerSignature On

RewriteEngine on
RewriteMap LeechProtect prg:/usr/local/cpanel/bin/leechprotect
RewriteLock /usr/local/apache/logs/rewrite_lock

UserDir public_html

# DirectoryIndex is set via the WHM -> Service Configuration -> Apache Setup -> DirectoryIndex Priority
DirectoryIndex index.html.var index.htm index.html index.shtml index.xhtml index.wml index.perl index.pl index.plx index.ppl index.cgi index.jsp index.js index.jp index.php4 index.php3 index.php index.phtml default.htm default.html home.htm index.php5 Default.html Default.htm home.html

# SSLCipherSuite can be set in WHM under 'Apache Global Configuration'

SSLPassPhraseDialog builtin
SSLSessionCache dbm:/usr/local/apache/logs/ssl_scache
SSLSessionCacheTimeout 300
SSLMutex file:/usr/local/apache/logs/ssl_mutex
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin

# Defined in /var/cpanel/cpanel.config: apache_ssl_port
Listen 0.0.0.0:443
AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl .crl

AddHandler cgi-script .cgi .pl .plx .ppl .perl
AddHandler server-parsed .shtml
AddType text/html .shtml
AddType application/x-tar .tgz
AddType text/vnd.wap.wml .wml
AddType image/vnd.wap.wbmp .wbmp
AddType text/vnd.wap.wmlscript .wmls
AddType application/vnd.wap.wmlc .wmlc
AddType application/vnd.wap.wmlscriptc .wmlsc

SetHandler server-status
Order deny,allow
Deny from all
Allow from 127.0.0.1

# SUEXEC is supported

Include "/usr/local/apache/conf/includes/pre_virtualhost_global.conf"
Include "/usr/local/apache/conf/includes/pre_virtualhost_2.conf"

# DO NOT EDIT. AUTOMATICALLY GENERATED. IF YOU NEED TO MAKE A CHANGE PLEASE USE THE INCLUDE FILES.
NameVirtualHost 174.121.25.14:80
NameVirtualHost *
# DO NOT EDIT. AUTOMATICALLY GENERATED. IF YOU NEED TO MAKE A CHANGE PLEASE USE THE INCLUDE FILES.

ServerName warawiri(dot)com
DocumentRoot /usr/local/apache/htdocs
ServerAdmin email(at)warawiri

suPHP_UserGroup nobody nobody

# Default vhost for unbound IPs

ServerName warawiri(dot)com
DocumentRoot /usr/local/apache/htdocs
ServerAdmin email(at)warawiri

suPHP_UserGroup nobody nobody