Lately I have been trying to get my server to be better at performance. So far it has been going well, but the beast currently as many people know is apache. The server im running drupal on is only running drupal. So my goal is to trim it down as much as possible and only have the apache modules I need to run drupal. I am alternately considering installing nginx as it is much lighter than apache. I have been reading the drupal performance articles on 2bits.com and it has helped but i have not found many articles on trimming apache this much.
Are there any real trims that can be done other than disabling a few apache modules?
Im basically looking for any articles, ideas or tips on or near the subject. For either nginx or apache.
Any ideas are much appreciated.
Here's what i'm Running:
Ubuntu Server 8.10 VPS
Ram: 512 MB
Processor: AMD Turion(tm) X2 Dual-Core Mobile RM-72 @ 500Mhz each
Comments
Have you installed APC or the
Have you installed APC or the like? Where's the bottleneck? CPU? RAM? Code execution time?
Apache MPM Worker + fcgid
Go for a threaded server, and PHP as an fcgid. There is an article on 2bits.com for that.
Disable the Apache modules that you don't need.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Disable the Apache modules
Do you have a list of Apache modules required by Drupal? Looks like the default install on my VDS has way too many enabled.
I've searched all over for the minimum requirements, as it seems dropping the un-needed fluff would be the 1st step to performance tuning.
This is what I usually setup
This is what I usually setup the
/etc/apache/mods-enabled/directory to be at a minimum, YMMV:alias.confalias.load
deflate.conf
deflate.load
dir.conf
dir.load
expires.load
headers.conf
headers.load
mime.conf
mime.load
php5.conf
php5.load
rewrite.load
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
Hi, I tried what you
Hi, I tried what you suggested here and now Apache won't restart. I get this weird error:
Syntax error on line 161 of /etc/apache2/apache2.conf:
Invalid command 'Order', perhaps misspelled or defined by a module not included in the server configuration
And lines 160-164 in the file are:
Order allow,deny
Deny from all
Satisfy all
I'm running Ubuntu 10.04LTS on a Linode.
JCL
I needed to: sudo a2enmod
I needed to:
sudo a2enmod authz_host
JCL
Can you specify some Apache
Can you specify some Apache handlers? It may more helpful for shared hosting.
Is it okay to run MPM Worker?
@kbahey: Thanks for the info. I am also struggling with an apache server that sucks up 150 - 250+ MB of RAM per instance about would love to try out MPM worker. I read this post on New Relic and got a little worried. Can you add your thoughts with respect to a drupal deployment, which is obviously PHP intensive:
https://docs.newrelic.com/docs/php/php-and-the-apache-worker-mpm
Apache2 MPM Worker + PHP-FPM
If you want to use Apache MPM Worker (threaded server), then you have to use PHP as a CGI, NEVER as mod-php5 (which is not recommended by the PHP team, since PHP is not thread safe).
We started by using fcgid with Apache MPM Worker, but this has the drawback that each PHP process has its own APC cache. Currently, we use and recommend Apache MPM Worker with PHP-FPM, where the APC cache is shared.
This way, the static files are served by a lightweight Apache thread, not carrying all the bloat of the full PHP inside it.
This is the old article about Apache threaded with fcgid
http://2bits.com/articles/apache-fcgid-acceptable-performance-and-better...
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
hmm... not much RAM. I would
hmm... not much RAM.
I would start with general and mpm settings, one of vps with few Drupal sites (768 Mb RAM) is running:
KeepAlive On
KeepAliveTimeout 2
Timeout 30
StartServers 4
ServerLimit 8
MaxClients 32
MinSpareThreads 4
MaxSpareThreads 16
ThreadsPerChild 4
MaxRequestsPerChild 1000
Of course they are not perfect, but in my case much better than default.
If you have ~200-250Mb left for Apache only, but serving some decent trafic, I would suggest to google "low memory Apache" and go with php as Apache module (dso) and worker/event mpm. Yes, I am not recommending prefork module.
In most cases every caching layer requires extra RAM, so select only those with the greatest impact or those, which substitute Apache.
Most of the enabled by hosting providers Apache modules are pretty lightweight or necessary for Drupal to run (safely) as it is more complex than "hello world" scripts on which some eye-popping general bechmarks are made, exept if you are running out ram and start to swap hard disk.
Another great way to boost site performance is to... optimize front end, such as minimizing http requests per page -> lessening quantity of tasks for web server -> less threads/processes/started servers. YSlow tool is very handy. Done that?
Also, outside Apache it surprised me, that disabling MySQL engines you don't use, like bdb, inndb saves another ~10-30Mb.
p.s. on what kind of server your VPS is running? CPU you mentioned is made for notebooks, am I wrong? Just 500Mhz??? Very strange.
drupal+me: jeweler portfolio
MySQL on another server...
I would highly recommend running you're MySQL database instance on another server. MySQL runs better with more memory thrown at it, so I would also recommend setting up TmpFS as well. Use MySQLTuner too to help find possible variables that could be cleaned up.
MySQLTuner:
http://www.howtoforge.com/tuning-mysql-performance-with-mysqltuner
MySQL TmpFS Configuration:
http://it.planet.mysql.com/entry/?id=21096
If moving your MySQL to
If moving your MySQL to another server, be sure that you have a fast connection between the two machines. I'm currently experiencing problems with MySQL running over a network which I'm yet to fully solve. What I did find though was that a very good way to speed up database access is to replace drupal's database caching with memcache, using the cacherouter module. This greatly reduces the number of database calls per page and really helps take the strain off your database.
--
Tom
www.systemseed.com - drupal development. drupal training. drupal support.
I do absolutelly agree - stay
I do absolutelly agree - stay on 1 box as long as you can, for example RAM is the cheapest resource, so double it and get memcached/varnish/APC or whatever ;)
drupal+me: jeweler portfolio
More info on my server: My
More info on my server:
My server is setup as a VPS through Slicehost using xen. My server is rated at the 512 option. As far as I remember the server has 4gb ram and a quadcore 2ghz processor. that server only has 512 VPS's on it so it is split up 8 ways between the VPS's (leaving each with 512 ram and 500mhz alloted to each server slice). all of their servers are split up evenly to ensure the slices are not oversold. The ram is set as 512 but is burstable to higher levels say for instance if i was dugg and as long as all the other slices had the room to spare. Though thats only allowed in a limited fashion and will ask you to buy a larger slice to accomadate your needs. Because of how its setup im not limited to what my hosting provider has, as when i get a new server slice I install the software I want and/or need so I do not have to worry about what my shared host is running. I have full root access and can wipe the server and start over any time.
As for APC, I have the zend zerver ce installed which comes with zend optimizer. I do have memcache installed on my server as well but i recently installed it and havent had a chance to get it properly tuned as of yet. I havent installed any other caching systems so far.
So far the closest bottleneck seems to be ram, in general the cpu response seems to be fairly good. Because of the smaller amount of ram i also have to make sure my mysql is tuned properly which i am hoping i have somewhat close (I will attach my *.conf files below). So far I have not hit swap yet.
@kbahey I have been reading
@kbahey I have been reading the articles @2bits and they have been really helpful. Though using the zend ce server im not sure if i can use the fcgid method (unless there is another method to do it with zend server ce, i havent had a chance to look it up yet but will when im done posting this). As for apache modules i dont need i have disabled the ones that im not using just for drupal. I also use it as a file server so I do have a few authz* mods enabled to password protect a separate domain using an htpasswd file and cookies. im also not sure what all modules i do not need as i havent looked them all up yet in regards to drupal needing them or not. below is a list of modules that are enabled for apache.
auth_basic.load, authz_default.load, authz_user.load autoindex.load, env.load, mime.load, negotiation.load, php5.load, alias.load, authz_host.load, cgi.load, deflate.load, dir.load, rewrite.load, status.load, setenvif.load, status.conf, authn_file.loadAfter reviewing the apache settings above I see that some of them are off. Here is what i had and changed their values to. I have adjusted them and are reflected in the attached conf file as well:
KeepAliveTimeout 10 -> 2
<IfModule mpm_prefork_module>
StartServers 5 -> 4
MinSpareServers 5
MaxSpareServers 10
MaxClients 10
MaxRequestsPerChild 0 -> 100
</IfModule>
<IfModule mpm_worker_module>
StartServers 2 -> 4
MaxClients 10 -> 16
MinSpareThreads 25 -> 4
MaxSpareThreads 75 -> 12
ThreadsPerChild 25 -> 4
MaxRequestsPerChild 0 -> 100
</IfModule>
so far as mysql goes I am only using myisam. i have skip-bdb, skip-innodb, and skip-federated all enabled. I would like to run a DB server but until i have more moolah to put towards my server it will need to stay on one server. If i want more ram i would need to upgrade my server slice to a larger plan or get a second server, which I could do but again funds. Upgrading a slice gives you the whole shebang though. Ram, HDD space and processor. they have all the servers pre allocated and not sold like shared hosts so its an all or nothing upgrade. In regards to the TMPFS i have looked it up briefly and will try it out. I did use a mysql tuning app found here: http://blog.mysqltuner.com/
Im aware that instead of using the built in drupal cache to use memcached and/or apc instead. Again apc in built into zend server ce as zend optimizer and i have memcached installed as well but have not been able to find a configuration for my size of server as all the ones i found assumed an 8gb ram server (which i obviously dont have). I have also looked into the boost module and tried it and works pretty well, but as the site isnt live yet im just not using it yet so I can make sure I can develop everything properly. I have also looked at the cacherouter module but havent been able to get it configured yet so it is disabled.
If I change php to run as an apache module instead then what are the drawbacks of doing this? how would running php fcgid with apc compare in performance to zend server ce with optimizer?
Apache MPM worker + fcgid
You already have PHP as an Apache module, since you have php5.load there. No?
The disadvantage of having PHP as an Apache module are:
1) to do this, you have to run Apache as pre-fork, which means that the PHP interpreter, and all the memory allocated by PHP scripts (i.e. Drupal modules, themes, ...etc) will be all in memory for the lifetime of the Apache process.
2) The process will have the large size even if it is serving a flat file.
3) For a busy site, yo have to have many processes (MaxClients), and given the above, most of the Apache processes will be large in size (e.g. 500 processes X 30MB) and hence memory utilization will be a lot. This causes the bloat you have seen.
Contrast that with running Apache as a threaded MPM Worker server, and hence the size of the process will be from 6 to 12 MB in size, with many threads within the the process (25 or so, whichever you set it too). These processes can serve both PHP content as well as the static files. PHP will run as fcgid, and you will have a lot less PHP processes than you had under pre-fork (normally just 5 to 10, and peak at 30 or 50, but never in the 100s).
So, to get rid of memory bloat, switch from mod_php + MPM pre-fork to fcgid + MPM worker.
Under this setup, a single 4GB server can run a site that gets a million page views a day and have spare memory too.
I hope it is clearer now.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
What about APC under this config (fcgid + MPM worker + APC)
Pardon my ignorance, in particular regarding mpm worker, but I am confused about the recommendation here.
It is my understanding (from 2bits.com detailed reports and other sources) that APC is not optimal under fcgid.
See "caveats and preacution" section at:
http://2bits.com/articles/apache-fcgid-acceptable-performance-and-better...
What's best (in general) for a multisite installation (24 sites) with significant combined traffic (1.5 million page views a month):
a) mod_php + apc
b) fcgid + apc
c) fcgid + mpm worker
d) fcgid + mpm worker + apc
and e) I am assuming that adding memcached to any a, b, c, d would improve performance, am I right?
(in my case it's 2 Dual Core Pentium servers each with 2Gb RAM, each with their own IP behind a load balancing ip running drupal 6 on a shared drupal folder installation created via symlink and both accessing a dedicated mysql server on a different machine in the same rack).
Thanks for any insight or clarification.
2bits and kbahey rock with their detailed reports and instructions, but I am still too obtuse to call the shots on my own without reading them over and over and comparing notes...
con paciencia y calma,
sube un burro a una palma.
fcgid
1.5 million means around 50,000 page views per day, which is not low, but not high either. Still, tuning can benefit such an install.
With fcgid, the big gain is that you end up using less PHP process than you would with mod_php. However, in fcgid, APC cache is per-process. If you use fcgid, then use MPM worker, because this makes Apache able to use many threads and save memory compared to pre-fork.
The good thing about multisite, is that the codebase is shared and hopefully APC will cache just one copy of the code, even though you have 24 sites.
What is the reason you have two servers rather than just one beefier server? Is it redundancy or load? You have to make sure that you share the minimal filesystem across the 2 server possible, because the more directory hierarchy you share, the more work the servers have to do, and it can be a bottleneck.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
About My Setup and Doubts on optimal Apache configuration
Thanks for your quick response.
The traffic of my sites is actually 10 million pages a month (3 million visitors, only about 100,000 Drupal registered users, detailed traffic on http://www.quantcast.com/p-feM7LmXBuZGuM ). But it doubles on December thanks to a popular Christmas site sharing holiday season traditions, recipes and stories from around the world. :)
I have been slowly moving my sites to Drupal (and by slowly I mean over the past 4 years), having succeeded in migrating the virtual communities and interactive components (mostly forum and blog) to a Drupal extension of the old ASP sites. So, for BibliotecasVirtuales.com (a static, ASP based site), I have set up portal.BibliotecasVirtuales.com (a Drupal based interactive site with forum and blog, etc.) and so on.
So, last year when traffic started to build up and the sites started to become slower I did what any newbie would do... I threw more hardware at it, wishing I could cover with it for my lack of time and experience in optimizing Linux heavy load installations.
I ordered a load balancing solution from my hosting providers (Bocacom.net) and they offered two load balanced servers with their own IP, behind a shared IP that distributes the balance. Only while migrating, I realized that I had a problem with this approach, since users upload files, etc. and after considering my options I took the easier approach to symlink the Drupal directory (public_html and all) of one server to the other. They both are in the same rack and so is the dedicated MySQL server I am using for the databases.
I hope that answers your question about the pure judgement behind my current setup :-)
Now, back to your answer...
Would you recommend I do (x) fcgid + mpm worker and no APC, (y) plain mod_php + APC, or would (z) fcgid + mpm worker + APC work fine?
I ask, because above you recommend "to get rid of memory bloat, switch from mod_php + MPM pre-fork to fcgid + MPM worker.", but make no mention of APC and hence my doubts.
Thanks for your prompt response and valuable comments and posts...
con paciencia y calma,
sube un burro a una palma.
Apache + mod_php + APC + Varnish
We're running about 30k hits/day on mod_php with APC and fronting it all with Varnish (and Pressflow). It's working well enough, but we're looking seriously at shifting off Apache and onto lighttpd or nginx to reduce the RAM needed for the web server processes.
Thanks for sharing, will follow your advice
I am planning to upgrade tonight to mod_php + APC + memcache.
I have not looked into Varnish yet as it is less documented and less commented among the Drupal community and there is more support by other modules to the other options (eg. Cache Router, Memcache API and Integration, APC - Alternative PHP Cache (for Drupal 7), etc.).
con paciencia y calma,
sube un burro a una palma.
don't forget about boost
if your looking at varnish, you should also look at Boost
Why? You shouldn't choose
Why? You shouldn't choose boost over varnish.
Boost has its uses. It's
Boost has its uses. It's simpler to configure, and can be of use on shared hosting accounts, RAM-limited VPSes, and other cases where Varnish installation is impossible or unfeasible.
The Boise Drupal Guy!
And about php handler?
I assume suexec is a no go with APC, I choose DSO and not fastcgi... let's see how it goes.
con paciencia y calma,
sube un burro a una palma.
Hard to recommend
What I can tell, is that a 4GB 4-core single Xeon can happily serve a site that does above 1 million page views per day (260,000+ visits per day). The server load average is between 1 and 2 and rarely goes up to 3. Only about 1 GB of memory is used. Memcache is installed with a simplistic setup of only 3 bins (one for page, one for filter, and one for everything else). I plan a detailed article on such a setup and what areas of tuning we did.
It is hard to recommend anything specific without really monitoring the server and identifying the reason(s) for the slowdowns, and see the resource utilization. You never jump to a treatment without diagnosis first.
Do not start changing configurations based on what you read (on our site or elsewhere). It is like trying every medicine on the shelf, or having one surgery a day because you heard about it. Do it methodically with observing, analyzing, and then implementing.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Sane Advice :)
I will share a detailed post with outcomes and follow-up
con paciencia y calma,
sube un burro a una palma.
Only 2GB of RAM?
Only 2GB of RAM seems really tight to me - especially if you're running APC and Memcache. Choosing between the various config options is going to have far less effect than simply giving the servers more RAM. It's all well and good to set up APC/Memcache/Varnish/whatever but if there's not enough RAM available to actually hold anything, it's not going to make much of a difference.
Server stats show plenty of available memory
With my current setup (fcgid+eaccelerator, although not optimized or tweaked, so probably the caches not even working for the drupal sites) the servers (including the database server) show no shortage of memory. I do have high peaks of cpu load (up to 20 times the optimal load to be handled by the servers) at regular intervals...
Load Balanced Server Memory (1)
PhysicalMemory
Total: 2074180
Free: 675796
VirtualMemory
Total: 4192944
Free: 4192944
Load Balanced Server Memory (2)
PhysicalMemory
Total: 2074180
Free: 946684
VirtualMemory
Total: 4192944
Free: 4192944
MySQL Dedicated Server Memory (4Gb)
PhysicalMemory
Total: 4151816
Free: 1289344
VirtualMemory
Total: 2096440
Free: 2096440
If it turns out that more memory if necessary, I have no issue with increasing it, but this time, I'm trying to do things right and not throw more hardware to the problem before optimizing the setup :)
Thanks for the advice though, and I will surely monitor memory usage with the new setup...
con paciencia y calma,
sube un burro a una palma.
An Update on My Current Setup
1. Switching from SuPHP+suexec to dso.
My load balanced servers came with cpanel/whm installed... which defaulted to suphp and had all files owned by the account under which they were created. When I attempted to switch away from suphp in order to properly use APC, I got permission errors for file upload directories, etc.
I fixed this by changing ownership of all "files" upload directories and uploaded files to nobody.
I then changed the php handler to dso and turned off suexec.
2. Enabling APC.
I then proceeded to download and install APC and added apc.shm_size = 64 and apc.rfc1867 = 1 to php.ini.
And it started to work apparently fine...
I was so excited monitoring it work via the included apc.php file...
2. Configuring Cacherouter for APC and then without it again.
So I went and changed the cacherouter module engine entry in settings.php from 'db' to 'apc'. Got a big scare when pages from one site started to show in other sites... but that was just because I forgot to provide different prefix entry for each of site as well.
Fixed that and everything looked fine.
I then proceeded to edit settings.php for the rest of the sites, including engine='apc'... and before I could react, the servers started to return memory overflow errors and became slow, until they finally stopped responding.
I suspect one of two reasons:
1) I was adding the cacherouter module lines in settings.php (which redefine cache_inc value) before (and without) enabling the module for many .
2) Perhaps, since each site had a different prefix, a new cache was created for each site in APC, so that would be 128Mb caches for 24 sites, well above the 2Gb physical memory of each server.
I had to reboot the servers, comment the cacherouter module lines in settings.php in those sites where I had not enabled the cacherouter module yet, and also returned the module's engine setting to engine='db' for those sites that had the module enabled.
After that , servers seem to be working fine with no unusual load and I have seen no more memory errors.
Regardless of setting cacherouter's engine to 'db', APC appears to be doing a great job.
You can check the stats for both servers (remember, sites are served by two load balanced servers under a common ip):
http://easycaptures.com/8534420241
http://easycaptures.com/5206845309
Hit rate is above 90% (it's actualy 99.9% as I write this) and memory consumption less than 100Mb (84Mb as I write this).
Since APC is working fine, I think I will not enable 'apc' as the engine in cacherouter and will let them both do their respective jobs.
3. Ocassional Server Crash.
I have experienced twice today one of the servers going offline (not responding via apache, shell, ftp, etc.). Users would not notice if it is not the one holding the file, since the load balancing will then route to the other one. But the server refuses to respond and I have to do a cold reboot (and even manually restart apache when it comes back up) to get it responding again.
I recall reading about such problem a few days ago and ways to fix it or at least recover from it. There was a script somewhere out there I think. Will look for it tomorrow.
4. Next Steps.
I asked for advice regarding some difficulties installing memched and for removing cpanel/whm from my servers, and my hosting provider (Bocacom.net) adviced we do a clean install (the technician didn't like some old libs and stuff on the server either). They have enabled 2 brand new servers for me at no cost (they will even cost US$50 less a month than the old ones) and I have until the 7th to configure them and migrate there.
I will ask for the price of upgrading to 4Gb each of the 2 servers (currently 2Gb each) and do it right away or next month, depending on my budget :p
Thanks for all the help and suggestions.
I will generate detailed notes of the new install and share them as a separate post later hoping they might help someone and also to get feedbacks and questions.
Regards,
Carlos.
con paciencia y calma,
sube un burro a una palma.
well i tried to attach the
well i tried to attach the files but no matter what i uploaded it would get marked as spam. i tried for nearly 25 minutes to no avail. I even had a difficult time submitting my two above comments. perhaps the restrictions are too strict on the mollom or spam modules. it is a little irritating. anywho here are some of my settigns:
apache:
Timeout 30
KeepAlive On
MaxKeepAliveRequests 150
KeepAliveTimeout 10
<IfModule mpm_prefork_module>
StartServers 4
MinSpareServers 5
MaxSpareServers 10
MaxClients 10
MaxRequestsPerChild 100
</IfModule>
<IfModule mpm_worker_module>
StartServers 4
MaxClients 16
MinSpareThreads 4
MaxSpareThreads 12
ThreadsPerChild 4
MaxRequestsPerChild 100
</IfModule>
mysql:
connect_timeout = 10interactive_timeout = 45
join_buffer_size = 2M
key_buffer = 20M
max_allowed_packet = 1M
max_connect_errors = 10
max_connections = 50
max_user_connections = 50
myisam_sort_buffer_size = 64M
query_cache_limit = 1M
query_cache_size = 4M
query_cache_type = 1
read_buffer_size = 2M
sort_buffer_size = 1536
table_cache = 1M
thread_cache_size = 60
tmp_table_size = 16M
wait_timeout = 45
command 'lshw':
*-coredescription: Motherboard
physical id: 0
*-memory
description: System memory
physical id: 0
size: 520MiB
*-cpu:0
product: Dual-Core AMD Opteron(tm) Processor 2212
vendor: Advanced Micro Devices [AMD]
physical id: 1
bus info: cpu@0
width: 64 bits
capabilities: fpu fpu_exception wp tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt x86-64 3dnowext 3dnow rep_good pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy
*-cpu:1
product: Dual-Core AMD Opteron(tm) Processor 2212
vendor: Advanced Micro Devices [AMD]
physical id: 2
bus info: cpu@1
width: 64 bits
capabilities: fpu fpu_exception wp tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt x86-64 3dnowext 3dnow rep_good pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy
*-cpu:2
product: Dual-Core AMD Opteron(tm) Processor 2212
vendor: Advanced Micro Devices [AMD]
physical id: 3
bus info: cpu@2
width: 64 bits
capabilities: fpu fpu_exception wp tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt x86-64 3dnowext 3dnow rep_good pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy
*-cpu:3
product: Dual-Core AMD Opteron(tm) Processor 2212
vendor: Advanced Micro Devices [AMD]
physical id: 4
bus info: cpu@3
width: 64 bits
capabilities: fpu fpu_exception wp tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt x86-64 3dnowext 3dnow rep_good pni cx16 lahf_lm cmp_legacy svm extapic cr8_legacy
mysqltuner results:
-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.0.67-0ubuntu6
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated -InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 8M (Tables: 201)
[!!] Total fragmented tables: 1
-------- Performance Metrics -------------------------------------------------
[--] Up for: 26s (700 q [26.923 qps], 236 conn, TX: 101K, RX: 116K)
[--] Reads / Writes: 99% / 1%
[--] Total buffers: 50.0M global + 4.4M per thread (50 max threads)
[OK] Maximum possible memory usage: 270.3M (52% of installed RAM)
[OK] Slow queries: 0% (0/700)
[OK] Highest usage of available connections: 2% (1/50)
[OK] Key buffer size / total MyISAM indexes: 20.0M/1.7M
[!!] Key buffer hit rate: 52.8% (53 cached / 25 reads)
[!!] Query cache efficiency: 0.0% (0 cached / 459 selects)
[OK] Query cache prunes per day: 0
[OK] Temporary tables created on disk: 20% (20 on disk / 97 total)
[OK] Thread cache hit rate: 99% (1 created / 236 connections)
[OK] Table cache hit rate: 97% (219 open / 225 opened)
[OK] Open file limit used: 0% (438/65K)
[OK] Table locks acquired immediately: 100% (240 immediate / 240 locks)
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
MySQL started within last 24 hours - recommendations may be inaccurate
Enable the slow query log to troubleshoot bad queries
Variables to adjust:
query_cache_limit (> 1M, or use smaller result sets)
yes, spam settings are way
yes, spam settings are way too strict, I had the same problems you mentioned.
I do not know your Drupal application, but increase MaxRequestsPerChild to 500-3000, also query_cache_size should be higher, unless you are serving very small database (not Drupal), I would suggest 10-20M, your mysql uptime is pretty small, so it's not known what is optimal query_cache_limit size, but I think it is ok 1-2M
sort_buffer_size = 1536 - it is some kind of a mistake? minimal value is 32K, optimal in your case - 32-256K
also questionable:
join_buffer_size = 2M - too big, or are you running some module with lots of heavy joins?
table_cache = 1M - supposed to be integer, not sure if mysql understands, but 1024 is ok...
read_buffer_size = 2M - absolute overkill and waste or RAM, 32-256K is more than enough, even fastest settings ususally are in this range.
for DB optimization during cron runs there is even Drupal module, forgot its name...
below it is (768RAM VPS) part of my.cnf, also not very optimal, just for reference...
[mysqld]
skip-networking
skip-innodb
skip-bdb
log-slow-queries
safe-show-database
thread_concurrency = 8
max_delayed_threads=20
max_tmp_tables=32
tmp_table_size = 32M
max_heap_table_size = 32M
query_cache_type = 1
query_cache_size = 48M
query_cache_limit = 2M
thread_cache_size = 32
max_connections = 40
max_user_connections = 40
key_buffer = 8M
key_buffer_size = 16M
record_buffer=16M
myisam_sort_buffer_size = 1M
join_buffer_size = 128K
read_buffer_size = 128K
sort_buffer_size = 128K
table_cache = 4096
open_files_limit = 12288
interactive_timeout = 25
wait_timeout = 1000
connect_timeout = 10
max_allowed_packet = 16M
max_connect_errors = 99999
drupal+me: jeweler portfolio
if your using private file
if your using private file transfer try http://drupal.org/project/xsend
Senior Drupal Developer at DrupalConnect
Script to Monitor Apache Memory Usage
The following php shell script helped me tune the process and memory parameters on our server. It displays the amount of memory Apache is using as a percentage of the amount of memory installed, the number of apache processes running and a list of the processes that are using more than a constant amount of RSS memory. You might want to "watch ./this_script" run for a while on your box or even create a mrtg page. Perhaps it will be helpful to you as well.
#!/usr/bin/php<?php
$fat_task = 20000; # large processes to monitor
# how much physical memory is installed
$str = exec('cat /proc/meminfo | grep MemTotal');
preg_match('/(?P<name>\w+): .* (?P<digit>\d+) kB/', $str, $matches);
$memory_installed = $matches['digit'];
# list the active apache processes and how much memory they are using
exec('ps aux --sort:rss | grep apache', $ps);
$pslines = sizeof($ps);
$total_mem = 0;
$total_tasks = 0;
print "\n Apache Processes with RSS > $fat_task:\n\n";
# for each of the lines in the output of the ps command
#
for($i=0;$i < $pslines; $i++) {
# only records that are user=apache
#
$first_sp = stripos($ps[$i],' ');
$user = substr($ps[$i],0,$first_sp);
if($user != 'apache') continue;
# find memory percentage field and print if "fat"
#
$mem = trim(substr($ps[$i],32));
$mem = substr($mem,0,stripos($mem,' '));
$total_mem = $total_mem + $mem;
$total_tasks++;
if($mem > $fat_task)print " $ps[$i]\n";
}
$total_tasks--; # less one for the grep task
$memory_percent = round(( $total_mem / $memory_installed ) * 100);
# print memory information specifically for apache tasks
#
print "\n $memory_percent% of System Memory is being used by $total_tasks Apache Tasks.\n\n";
# let's see what is happening with memory in general
#
print "\n free -m:\n\n";
exec('free -m',$free);
for($i=0; $i < sizeof($free); $i++)print " $free[$i]\n";
?>
Great to have this as a
Great to have this as a script. A few points:
watchdoesn't work as the list of processes is too long.--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
Thanks for the feedback
I should have mentioned that I am running a RHEL 5 system with Plesk -- thus the 'apache' process owner. This script uses systems calls that I'm sure are not exactly the same from OS to OS. If I had to guess, there are likely some slight formatting differences of meminfo or ps output which is causing the "math error".
It sounds like you have a large number of processes with RSS Memory > 20K. You might want to increase the value of $fat_task which will reduce the number of tasks you are 'watching'. I found watching to be very useful in setting the KeepAlive parameters of Apache.
This script helped me "see" what was happening with my Apache memory. I hope it will be helpful to some as they work to tune up Apache. Please feel free to add a couple of print statements to flush out the cross system issues.
Enjoy! Greg
sometimes the exec or
sometimes the exec or shell_exec function is not allowed on some hosting providers
Slow performance
Hi there,
After reading many forums, still not able to get the best result:
My office is using Drupal 6 for our intranet, we use SUSE LINUX 11 on VMware.
During peak, we have around 400+ concurrent users accessing the intranet, the server
has been hammered and users are complaining the latency problems.
I've 4GB RAM and single core, I estimate average of 12MB per Drupal page request. It is not easy for me to change/add modules on server as the company has a stupid policy, doing those tasks only must be performed by our vendors, and to dealing with them is slow and painful.
The only quicker solution that I have is to tune the Apeche configuration and increase the RAM size tpo 8GB, wondering my prefork configuration needs to be modified.
# number of server processes to start
# http://httpd.apache.org/docs/2.2/mod/mpm_common.html#startservers
StartServers 5
# minimum number of server processes which are kept spare
# http://httpd.apache.org/docs/2.2/mod/prefork.html#minspareservers
MinSpareServers 5
# maximum number of server processes which are kept spare
# http://httpd.apache.org/docs/2.2/mod/prefork.html#maxspareservers
MaxSpareServers 10
# highest possible MaxClients setting for the lifetime of the Apache process.
# http://httpd.apache.org/docs/2.2/mod/mpm_common.html#serverlimit
ServerLimit 250
# maximum number of server processes allowed to start
# http://httpd.apache.org/docs/2.2/mod/mpm_common.html#maxclients
MaxClients 250
# maximum number of requests a server process serves
# http://httpd.apache.org/docs/2.2/mod/mpm_common.html#maxrequestsperchild
MaxRequestsPerChild 10000
12MB per Drupal page request
12MB per Drupal page request sounds little to me. I've seen Drupal actions running out of memory while there was a lot more available then that. What is your value for memory_limit in php.ini? According to http://drupal.org/node/207036, 16MB is the bare minimum for Drupal 6 and 64MB is recommended.
Also MaxClients 250 looks very high even when you have 400+ concurrent users.
Note that (MaxClients * memory_limit) will roughly give you an idea of how much RAM apache with mod_php will use. Make sure that amount of RAM is available and, depending on your setup, even a bit or a lot more for the rest of the OS, memchache, apc, mysql and other services to run.
Slow Performance - Seriously improved
Fine tuning performance is an art.
In our case the answer to significantly improve performance was to get the right combination of caching features in Drupal, Apache, MySQL and PHP to work efficiently.
Adding more RAM for Apache alone will not make a big difference.
What you could consider is increasing the :
- PHP memory
- MySQL query cache
Check and re-check your Drupal cache settings, especially for things like views, and panels if used. SSL and using the private file system in Drupal slows down a site a lot.
As mentioned before in this thread - the one thing that will seriously improve the performance and worth dealing with your "vendor" is to install Memcached on your server. This most certainly solve many performance issues. Most of the traffic to your Drupal site will be handled by memcached in RAM, and depending on your configuration, it could be as much as 90-98% of all requests.
We actually made a guide for How to install memcached on openSUSE for use with Drupal