Hello Everyone,
I have a website with 100k Visitors/Day , I am a newbie in Php and Servers, when a downtime occurs managed hosting guy was resetting the server and I was optimizing the databases and I was optimizing the DB everyday and I was fine.
But this week site became to crash twice a day. And Reboot/optimize only works for 5-6 hrs and it crashes again. My hosting guy said it is caused by cache_page table in mySQL and it gives ~300MiB overhead in 2hrs after mysql optimization. And he said its a wrong method to keep cache in database it causes this errors.
I had some research and found modules "Cache Router" and "Memcache API and Integration" but could not install them :( Also hosting guy does not anything about drupal. Also he installed memcache at 127.0.0.3:11211 as my request. But i could not figure out how to install these modules
It says this is a default code in cache router:
$conf['cache_inc'] = './sites/all/modules/cacherouter/cacherouter.inc';
$conf['cacherouter'] = array(
'default' => array(
'engine' => 'db',
'server' => array(),
'shared' => TRUE,
'prefix' => '',
'path' => 'sites/default/files/filecache',
'static' => FALSE,
'fast_cache' => TRUE,
),
);
But it does not make sense to me, where will I put my memcache server after changing engine to memcache? Please post a code for my needs if cache router will solve my problems.
I am running a default drupal site without any tweaks ( only Drupal normal caching is enabled ), I have 350k pageviews and 100k visitors Quad Core Xenon Server with 4GB Ram.
Please give your opinions, Thank you very very very much
Comments
Can you post a link to your
Can you post a link to your site? It could be helpful to see it in terms of which pages are getting highest visibility/traffic.
Here
www.animefreak.tv crashed 7 times in 2 days. It will be great if u can recommend me smt
Go with
Go with this:
$conf['cacherouter'] = array(
'default' => array(
'engine' => 'memcache',
'server' => array('127.0.0.3:11211'),
'shared' => FALSE,
),
);
However, running on a single server you would do better to go with APC, as that gives you opcode caching as well as a user caching mechanism cacherouter can use. If you can get your hosting guy to install APC for you, then you would just go with this:
$conf['cacherouter'] = array(
'default' => array(
'engine' => 'apc',
),
);
APC is actually faster than memcache, but will only work for object caching if you are on a single server, as it doesn't share data like memcache. If you ever have the need to go to more than one webserver, then you need to go back to memcache, but would still want to keep APC installed on the server for the benefit of the opcode caching, which alone can give up to a 25% performance increase in PHP.
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.
thank you very much, but i
thank you very much, but i read several articles today. What are the hit/miss ratio? Also I have to serve latest content to my users and they should see what content is added in 10 mins ago.
Is it bad using memcache in single server? And last thing: Both of these methods will solve my cache_page and cache_menu table overhead problems? Because he says these tables are the reason for crashes.
Thank you for your time for helping this newbie, Also i wish to see more recommendations from other experts
Hit/miss ratio is basically
Hit/miss ratio is basically the ratio of number of items that cache returns to the number it can't find.
For showing the latest content, go to admin->site configuration->performance and set your caching method to normal and adjust the cache lifetime to "none", then it will force the cache to refresh every cron run.
Memcache is fine on a single server, but you will get added performance if you used APC instead. If anything try memcache for now and see if that helps out. It will fix the overhead problem on your cache tables as those won't be used anymore. If you are still experiencing some problems or slowness, then I would switch over to APC since you also get the benefit of opcode caching with that. Without opcode cache, basically PHP has to load every file that makes up Drupal and compile it to opcode. With opcode cache, that doesn't have to occur anymore.
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.
$conf['cache_inc'] =
$conf['cache_inc'] = './modules/cacherouter/cacherouter.inc';
$conf['cacherouter'] = array(
'default' => array(
'engine' => 'memcache',
'server' => array('127.0.0.3:11211'),
'shared' => FALSE,
),
);
I installed cache router and modified settings.php, cleared the cache, repaired/optimized DB and ran cron. it worked fine for me in offline mode but when I changed to online it gave this error in 5 mins and server crashed
"The mysql error was: Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (11)." and the server load became 89.26, 72.81, 32.60
and now it crashed, give "page not found" error
Are you sure memcache is
Are you sure memcache is properly installed? The best way to check is through phpinfo. Just create a simple file and put:
<?phpphpinfo();
?>
In it and see what it reports. You should have a section in there for memcache.
You can also get memcache stats directly from the command line:
echo stats | nc 127.0.0.3 11211
That will give a bunch of data, like total connections, sets, gets, bytes written, bytes read.
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.
http://www.animefreak.tv/a.ph
http://www.animefreak.tv/a.php
No APC?
You don't have APC installed. Start with asking the hosting company to install it for you.
Don't use cache router yet. Just see what APC will do, then we can revisit data caching after we have op-code caching working first.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Khalid, Do you think APC is
Khalid,
Do you think APC is a worth the effort at this stage? It seems like some other areas might be more fruitful. I had wanted to see the front and main landing pages to see how much data is being called for. If that is high, then boost.module might be a good first step.
Couple of other things to check:
1. If Apache is configured with too few clients available (MaxClients value in httpd.conf) users could experience contention just trying to access the site. If it is configured too high, you might be asking too much of the DB, or asking for more DB connections than available.
2. Look at your drupal logs (or Apache) for 404 errors - i.e. file not found. Each 404 that get's through to Drupal will require an entire framework load just to serve the 404. We defend against this in .htaccess in the Drupal root directory by serving a custom static error page - code as follows:
# check for '.' in URL
RewriteCond %{REQUEST_URI} \..*$ [NC]
# file not found?
RewriteCond %{REQUEST_FILENAME} !-f
# then server custom static page
RewriteRule .* http://www.yourdomainname.com/file-not-found.html [L]
Definitely worth it
APC may not solve all the issues, but it is a prerequisite first step. Definitely with it. No large site should ever run PHP without an op-code cache.
It makes page generation times go down, makes Apache processes take less time per request (and hence can serve others more quickly) and reduces both CPU and memory utilization.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Need PECL memcache for PHP
http://pecl.php.net/package/memcache
Cheers
Run this in a terminal: echo
Run this in a terminal:
echo stats | nc 127.0.0.3 11211
And paste the output of it here, so we can see what the actual internals are doing. PHP is showing that memcache is installed, but that doesn't mean it's actually working.
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.
[root@server
[root@server memcache-3.0.0]# echo stats | nc 127.0.0.3 11211
STAT pid 23649
STAT uptime 43
STAT time 1237305691
STAT version 1.2.4
STAT pointer_size 32
STAT rusage_user 0.000000
STAT rusage_system 0.000999
STAT curr_items 0
STAT total_items 0
STAT bytes 0
STAT curr_connections 1
STAT total_connections 2
STAT connection_structures 2
STAT cmd_get 0
STAT cmd_set 0
STAT get_hits 0
STAT get_misses 0
STAT evictions 0
STAT bytes_read 6
STAT bytes_written 0
STAT limit_maxbytes 67108864
STAT threads 1
END
Memcache doing nothing
This shows memcache doing nothing at all. There are no cache sets nor gets, and hit and miss is zero as well.
First things first. Install APC, check things out then we move to memcache.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Boost module
Along with APC, you need to install the boost module. You'll probably gain a more significant benefit from that than any other single item.
im using url aliases, i
im using url aliases, i tried boost in my local server but it gave blank pages half of the content.
mysql max_allowed_packet
You may want to check this setting in your my.cnf file. We upped ours to 32M. The packet size when saving to the cache table was too small, and was causing similar problems ... This setting may also cause a problem when using phpMyAdmin. It may cause errors when importing/exporting Drupal DBs through the interface.