Hi,
I have been trying out various cache modules to bring the cpu load down on my drupal site.
I was having trouble seeing this work but started using devel performance monitor.
What is has shown me is that practically all my pages consume a 24-25 mb of memory.
However the milliseconds were chopped in half by the cache from 1351 to 518ms.
Actually the 1351 page was at 31mb and dropped to 25mb so the cache is doing a good job.
But also from looking at the performance it seems even the logging page comes in at 21mb.
Is this normal?
I assume drupal has a minimum payload, although I am guessing that it could be the PHP causing the 24mb.
Can I chop the Memory usage and CPU hit down more?
However I do admit to knowing very little about performance.
(I have little control over the server and don't have apc)
(I can't use boost because of theme switching despite it being awesome)
(I am aiming at anon users.)
Comments
What kind of caching are you
What kind of caching are you using here? I suggest looking at the advanced cache modul togheter with the memcache module. Both require you to hack core but it is worth it.
Thomas Hambach, Web developer
Don't hack core!
Ack! We use APC and Cacherouter and have had excellent results dropping pageload times. Neither require hacking core.
cough chicken cough
cough chicken cough
Thomas Hambach, Web developer
Bah!
Not chicken, smart. :-) When one runs as many sites as I do, the last thing I want to have to worry about is re-applying a bunch of patches to core every time there's a core update. Hacked core is a ticking time bomb just waiting to screw you up in the future. "Tomorrow you" will thank you for taking the time to not leave them problems.
Few things are worse than having to re-do a job because you didn't do it right the first time.
The only smart thing about it
The only smart thing about it seems to be that you try to exhaust your other resources before hacking core. Thumbs up to that.
Ticking timebomb? Come on... Don't you test you performance adjustments before you deploy?
Thomas Hambach, Web developer
I think you are missing the
I think you are missing the point of modular development...
I'm using memcache and views
I'm using memcache and views cache.
Also switched on the caching in mysql too.
Views cache has html caching and query cache.
I think the mysql side is cached pretty good but I need to reduce the php.
Just a guess though.
No APC?
To get back to your original question ...
Sadly, there's not much you can do to lower CPU if you can't run APC. Cacherouter supports memcache as well, but I expect if you can't run APC you won't be running memcached, either. Plus, memcache won't help lower CPU utilization nearly as much as a PHP opcode cache like APC, since an opcode cache not only caches content it also caches the pre-compiled PHP code.
Would it be possible for you fire up a caching reverse proxy like Varnish or Squid (preferably the former) in front of your Apache processes? That would certainly take some load off the CPU.
gchaix, I've compared D6
gchaix, I've compared D6 without memcache & advanced cache to D6 with memcache and did see a positive effect on CPU.
Ben, are you using a proxy such as squid or mod_cache yet?
Thomas Hambach, Web developer
Sounds like I need APC
Sounds like I need APC then.
The server I am on has memcached, because the CTO uses it on his production sites.
Basically, I have what he uses and has set up to optimise his sites.
But I can't add anything to the servers without permission.
We already discussed APC and in he has said that most of the weight comes from mysql and so doesn't need APC. Which means I can't use APC ( without getting into an argument).
He really means that APC doesn't do much for his sites....... I think it would be different for Drupal.
Is there is a sneaky way I can install it I would give it a go.
The same applies to proxy stuff.
I mentioned it but again I need permission to go and set it up, and I don't think he wants me running amok on his server.
So I am left to tweak with clever modules and memcache.
I tried boost and it was great but I had issues with theme switching screwing my mobile version of the site.
However I will probably look at that again.
Like you wrote above, I think I need to op the PHP side but don't really know how to go about without APC or
how to go about cunningly installing it with limited server access.
I am going to try and run APC on a local MAMP server just to see what the result is.
Hopefully I can then try and justify using APC without challenging the authority.
(if this sounds a bit weird... I work in Japan. Seniority system is way not dead, even if they say it is)
I want to get under 4 and maybe a kind of 2-4% if possible.
Boost got it down to 0.2% LOL! But I have already mentioned about boost.
To be fair ...
MySQL queries are probably more resource intensive (and certainly more time-consuming) than compiling PHP code at pageload. But I've yet to find a PHP site that didn't benefit noticeably from an opcode cache. It'll help his sites, too. :-)
That's a good point... Do
That's a good point... Do you have the mysql query cache enabled and configured?
Yup mysql query cache is
Yup mysql query cache is enabled and configured.
I have views-cache running for html and mysql as my site uses view a lot -- It did a good job.
I have the memcache module running too -- It also did a good job.
I have mysql query cache switched on and configured -- It made a bit of difference.
I think mysql is getting good coverage.
So I am assuming that the cpu/ram hit is from elsewhere, which is the PHP part.
When I used boost and it avoided the bootstrap or whatever magic it does it totally dropped the cpu hit (obviously!)
This is where I think the cpu hit comes from, and so was wondering what an average RAM load / cpu load for a drupal page is. And then how to reduce the load.
The final plan would be like boost and making static pages, but I still need some dynamics so looking for more optimisation tricks.
APC is the boi :p I put the
APC is the boi :p
I put the site on my MAMP server and did some runs.
MAMP also clocked the MB in at around 15mb dropping the cpu hit, so I guess the MAMP config must have some cool stuff running.
So then I switched on APC...... so nice :p
Pages down to 4-5mb.
I guess I have to figure out how to install apc on the test server.
It's a tricky one now because I had suggested APC and got told we didn't need it because it didn't make that much a difference.
So now I gotta suggest it without stepping on toes.
If you've got a lot of other
If you've got a lot of other sites with a lot of code on this box you may need to up the memory for APC. The default is I think 30MB. Use the APC control panel to see the current usage.
http://www.electrictoolbox.com/apc-php-cache-information/
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
20-30 MB here
Thanks for this discussion. Drupal performance is somewhat of a mystery to me. It's helpful to hear about other's numbers.
According to the Devel module, my pages weigh in at 20-30 MB each.
The server CPU got maxed out when we had 10k anonymous visitors after a TV spot (maybe only 500 at any one time). It scared me how poorly the site scaled. I scrambled to set up caching, installed the Boost module and that helped hugely.
I am now looking at installing APC although I'm not sure if it will give as much of a gain in addition to Boost, since the traffic is entirely anonymous.
It's true that APC doesn't
It's true that APC doesn't help with serving Boost cached pages to anonymous users, but it helps with everything else including caching a page for the first time, doing administrative tasks, cron runs, 404 errors, searches, etc. Depending on your content update frequency and your Boost settings, you might be regenerating those Boost pages periodically.
Unless you're running on a server with a tiny amount of memory, the 30-50MB that APC uses (depending on how many modules you have running) is going to be worth it.