Very Slow Caching, Memory Spikes

Events happening in the community are now at Drupal community events on www.drupal.org.
Ony's picture

Hello, it's me again. I've been trying to monitor my site for some days now and it occured to me to re-enable Devel to check out the speed of my queries. Turns out they're the problem. Some pages load faster than others, but for places like the forum, opening a link takes FOREVER, and it's not hard to see why, on one page below these were the longest running queries in the list. There way many more cache drags like the one below.

4755.013ms DrupalDatabaseCache::setP A
SELECT 1 AS expression FROM cache_menu cache_menu WHERE ( (cid = :db_condition_placeholder_0) ) FOR UPDATE
default

1291.731ms DrupalDatabaseCache::setP A
INSERT INTO cache_metatag (cid, serialized, created, expire, data) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4)
default

1068.3819ms DrupalDatabaseCache::getMultipleP A E
SELECT cid, data, created, expire, serialized FROM cache WHERE cid IN (:cids_0)
default

I'm already using Boost, which is great. And to be really fair, as a logged in user, once you open the pages a second time they DO load faster, but the visitor isn't going to know that. They're just going to encounter slow loading page after slow loading page and be turned off. Any suggestions as to how to get this to be less slow? Someone somewhere suggested backtracking through modules to see if maybe one of them was messing with my cache, but I have quite a lot running, and they were installed weeks ago now. I still don't have access to my.cnf so the db tuning seems out of the question.

Secondly, there's this really annoying thing happens when I publish posts most of the time. It causes a severe but short memory spike that causes my host to reboot my server. The essence is that if I publish a post, people that click through to it immediately, say, from Twitter, can't access the site for about 2 or 3 minutes. Is there a way to stop these memory spikes from happening? At any other time the memory usage is pretty decent.

Thanks

Comments

Check disk contention ...

kbahey's picture

One possible cause is slow disk performance. If your host is using Virtuozzo with the vfs file system, then disk performance can be slower than using Xen for example. Even Xen can be slow if the physical disk is shared by other virtual hosts that are hogging the disk.

Another cause is disk contention. Cache expiry in boost can cause this contention sometimes.

Look in those two possible causes and eliminate them before moving to other possible causes.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Thanks for the info. Is this

Ony's picture

Thanks for the info. Is this something I have to ask my host or is there another way of checking this. I'm on a VPS

Virtuozzo

kbahey's picture

You can ask your host if Virtuozzo is your server's technology. You can sometimes solve issues by asking them to move your VPS to another physical server. This works if there are other sites on your physical server that are hogging the disk.

As for boost, you can tell if it is the issue if this slow down happens after a cache clearing operation, e.g. posting a comment.

Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.

Agreed that Boost is probably

dalin's picture

Agreed that Boost is probably the cause of the slowdowns after you publish a post. It is very easy to configure Boost's cache expiry options in such a way that causes intense server load. But if your site is a forum (and perhaps focused on authenticated users), perhaps Boost isn't that useful to you to begin with.

As for the the slow cache queries, in addition to Khalid's suggestion about slow disks, I can think of three other possibilities:

  • General misconfiguration of MySQL. If you are getting similarly slow times for other simple queries it may be that the MySQL configuration doesn't meet the needs of your site. If you have access to change my.cnf I suggest using a tool like this to set a good starting point:
    https://tools.percona.com/

  • Tables using the MyISAM engine rather than InnoDB. InnoDB is the better choice for most web applications, especially for Drupal, especially for the cache tables.

  • General contention on the cache tables. Since there can be many simultaneous gets/sets to the caches from multiple HTTP requests the full features of a relational database tend to get in the way of these simple key/value queries. Using an alternative cache back-end can greatly improve things:
    http://drupal.org/project/memcache
    or if you are on shared hosting this one might work (but if disk access is your underlying problem this may make it worse):
    http://drupal.org/project/filecache

--


Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his

And if you are on a Virtuozzo

criznach's picture

And if you are on a Virtuozzo based VPS, and you have what they call "boost memory" (your host might call it something different), you need to pretend that doesn't exist to have a reliable Drupal site. Many hosts advertise this as a feature of their hosting - incorrectly IMHO. The base memory is all that's guaranteed. Once you start using the boost memory, Virtuozzo starts killing processes and other nasty things, depending on how the system is configured.

I am facing the same problem

tassaf's picture

I am facing the same problem .. I tried everything, but still the same
on a dedicated server

How about tuning up your

sabihahmed's picture

How about tuning up your entire website? There are several steps by which you can easily perform this option to have an optimize version of your website, and to get over from slow caching issue like you mentioned earlier.

High performance

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: