Serious serverload issue with Drupal
I'm having serous issue with private file transfer, I have about 50 thousand images added to the system and all transfer via drupal private file system. I have about 2lks users and when about 200 online, average server load about 30-40, its heavy load to server, I'm running my mysql server on localhost, what should I do to increase performance?. its taken about 20s to load a single page, about 20 mobule installed with views , cck etc.
I recently installed boost module but still the server load is heavy. What should I do?
Groups:
Login to post comments
Are you using drupal cache?
Are you using drupal cache? What version of drupal? 6? Also, are you using APC or something similar?
Yes, I tried Drupal cache
Yes, I tried Drupal cache but the site load was about 30-40, then moved to the boost module it worked to gain some performace abut still at the 25-30.
Also, what is apache's cpu
Also, what is apache's cpu usage? and what about mysql's?
Boost is not the solution
Boost/Core Cache will not help in this case most likely; it only works for anonymous users. APC would be a good place to start. Profiling is whats needed; you need to investigate why it's running so slow.
You meant about Cacahe
You meant about Cacahe router?
I didint try it, will that help me, I think most of the case will be due to private file transfer. In the main page loading about 50 images 130*130 thumbnails with imagecache, does file transfer show down the site and take a lot of resources?
You don't need Cache Router
You don't need Cache Router to benefit from APC. APC will cache the actual PHP files that makes up Drupal into op-code (machine readable code) , so that the server doesn't have to parse and compile the files on every request. If you have enough memory on your server, then you can use APC's object caching (not to be confused with op-code caching) to replace the database cache in Drupal. Op-code cache alone gives about a 20% performance boost, and then you get even more using the object caching.
One thing on Image Cache - are you sure the thumbnails are being written to the server when they are generated? If not then that would account for a huge performance impact. I would check this page and make sure everything is met:
http://drupal.org/node/224913
you need help? You need to
you need help?
You need to provide details
- which modules you use
- server configuration
-- php version/configuration
-- mysql version/configuration
-- web server name/version and configuration
"I have problem" - don't help us to image what is happened.
Did you try to install and setup devel module to track queries?
to get help you really need to spent more time and write good description about your situation.
Can we use both boost and
Can we use both boost and cache router module both active?
Yes
Yes, both can work together. Everything passes by boost first; it boost can not serve the page (not in cache, not anonymous user) then boost hands the page request off to Drupal, where cache router picks it up.
@heshanmw
Alexandr is right, "you really need to spent more time and write good description about your situation".
Give us the details: server CPU/RAM available, php/webserver etc. so we can help you diagnose and resolve the problem.
I would recommend replacing Apache with nginx but it will not help when you have not enough RAM or not optimized MySQL configuration.
You shouldn't try to find solutions when you don't know where is the problem.
~Grace
The problem with the private
The problem with the private file transfer is that it runs Drupal for each of your images. So, if you have a gallery page with 10 images, you will run Drupal 11 times to display that one page.
So, the best thing to do (if you need to use private file transfers) is to limit the number of files that show up on each page.
Wow. Are there any
Wow. Are there any alternatives to private file transfer that allow for restrictions to be placed on direct linking to images/videos?
I was doing some searching around, and came across this: http://drupal.org/node/74472
Does anyone have any experience with that patch?
Doing more searching, I
Doing more searching, I found this article discussing the same approach using lighttpd's mod_secdownload:http://redmine.lighttpd.net/wiki/lighttpd/Docs:ModSecDownload
There's a php example in it. Has anyone ever tried this with Drupal?
The underlying issue, as
The underlying issue, as Gerhard points out, is the number of Drupal bootstraps required to serve a single (HTML) page.
One thing that we have found can relieve this somewhat is to place a reverse proxy (in a non-caching configuration, at least for files you know are really private) such as Varnish in front of the web server. This allows Drupal to pass off the http request much much faster than it would if it were passing it to a remote internet client. This can result in much shorter request times, and a fewer number of active Drupal bootstraps, much reducing server memory usage (and freeing up memory for MySQL and disk caches).
definitely watching this
definitely watching this thread!
+1
---
naslenas.com. Drupal blog experiment.
First thing I would try is
First thing I would try is getting APC and Cacherouter setup. Yeah the private download is bootstrapping Drupal multiple times, but you might find that setting up those two items reduces the server load to the point that you can handle it.
This is good solution, but
This is good solution, but you will not get significant boost in performance with caching. Significant = 10-20x... So, there must be some sort of workaround.
---
naslenas.com. Drupal blog experiment.
What about this? this is
What about this? this is about the private file transfer, seems private file transfer is a bottleneck for performance http://groups.drupal.org/node/1043
finally i figured out the
finally i figured out the issue, it was mainly due to the private file system , i think there should be a good private file system wrapper, cause in my case i dont want private file system for thumbnails so I wrote a script to skip it, it worked and it took my site about 50% more performance,
Thanks all who support on this issue, looking forward for best performance in D7 :)
i have some problem with
i have some problem with APC, when I install APC its seems not active for the Drupal installation directory. Is there any special way to install it?
When you go to
When you go to yoursite.com/admin/reports/status/php is there an APC section. IS it enabled? What OS are you on?
@briansea : I cannot see it
@briansea : I cannot see it is on, do I need to use cache_router with this? I'm in FreeBDS
I installed memcached and
I installed memcached and still my server load is very high, i'm running one demon on the same server. My SQL also at the same server.
Server configurations are Apache 1.2
PHP, 5.2.10
MySql 5.2
OS: FreeBSD
RAM : 4GB
memcached is running well, but server load increasing upto 80, it will crash the server. What is the best thing that I can do, installing APC break my site, only white pages. I think APC not compatible with the FreeBSD.
Overhead is cause of mysql server, I'm using views module to display 50 images per page with several sorts(about 10).
--- Duplicate ---