APC + File cache + DB cache bin best practices

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
btopro's picture

http://dropbucket.org/node/1046

I was hoping maybe some one had an article or could give me some feedback on the following settings.php configuration that I'm currently using to push Drupal 7 cache bins to APC user cache and the file system via the File Cache module. Best performance numbers I got was APC for everything but that obviously has some issues with fragmentation of the smaller cached values. Some context of what I'm also using for performance
-- Display Cache https://drupal.org/project/display_cache
-- Entity cache https://drupal.org/project/entitycache
-- This system is a multi-multi site via DSLM https://drupal.org/project/dslm
-- Users of the system are almost 100% authenticated, primarily for viewing content though they will be submitting content in the future.
-- Server has 4 gigs of ram, 256 M currently dedicated to APC caching.
-- primed opcode cache sits at about 100 M

Currently most sites pull page loads (via devel stats) of between .15 and .25 seconds with minimal traffic (15-20 concurrent people on the box), we're just easing into the semester so I'll have a better idea of what that looks like when there's greater traffic. We have a theoretical max right now of around 1000 though the most we'll probably get at any one time is more like 50-80.

Any feedback about bin optimization would be appreciated.

Comments

What about memcache?

jpamental's picture

Hey Bryan -

What about memcache? Wouldn't that help with authenticated user performance?

Jason Pamental
[ @jpamental ]

APC provides the same basic

gp177's picture

APC provides the same basic functionality of memcache; moving cache out of the database and into memory. The benefit of memcache might be that it may handle many small items better then APC.

We are using memcache for all of our Drupal caching and use APC strictly for opcode caching without issue.

What we haven't done yet, and I'm interested in doing, is testing the new Drupal memcache module which also supports session storage.

https://drupal.org/project/memcache_storage

@gp177, You may ask me

Spleshka's picture

@gp177,

You may ask me directly any questions about using this module or its integraions.

Cool

perusio's picture

@Spleshka

Very nice indeed. With the other memcache module the key was really strange and it was not possible to replicate it on "normal" nginx. It required Lua to write the cache key.

Looking forward for testing it and updating my config with it.

Cool, would be great! I use

Spleshka's picture

Cool, would be great! I use your nginx config :)

With Memcache Storage you need to have a memcache key like this:

# With prefix option.
set $memcached_key "PREFIX-cache_page-$scheme://$server_name$uri$is_args$args";

# Without prefix.
set $memcached_key "cache_page-$scheme://$server_name$uri$is_args$args";

Although is works with page compression. So you may enable page compression in Drupal and set memcached_gzip_flag in the nginx config to 1 - and that's it!

And the basic settings.php settings for this:

# Move all cached data (except form cache) to memcache storage.
$conf['cache_backends'][] = 'sites/all/modules/memcache_storage/memcache_storage.inc';
$conf['cache_default_class'] = 'MemcacheStorage';
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';

# Advanced usage of Drupal page cache.
$conf['cache_backends'][] = 'sites/all/modules/memcache_storage/memcache_storage.page_cache.inc';
$conf['cache_class_cache_page'] = 'MemcacheStoragePageCache';

# Enable storing of plain HTML text instead of Drupal usual cache object.
$conf['memcache_storage_external_page_cache'] = TRUE;

I don't recommend to move

Spleshka's picture

I don't recommend to move drupal cache to the APC for the next reason: APC usually uses as an opcode cache tool. If you also move another cache to this memory section, then fragmentation will be poor and the performance will be lower.

My suggestion is to now mix up an opcode cache and drupal cache. If you want to store both caches in the memory, please use APC for opcache and Memcache for drupal cache.

Memcache uses LRU for

alanmackenzie's picture

Memcache uses LRU for eviction when it the cache is full where as APC will clear the entire contents of the cache.

Additionally you can't share access to an APC bin across multiple webservers so it's going to have poor performance when you scale up beyond a single machine for availability or capacity reasons.

I would strongly suggest only using APC for opcode caching and memcache for everything else.

Can't use nginx because of

btopro's picture

Can't use nginx because of sticking to RHL 6 64bit as well as lack of support for an internal central authentication system in nginx.

Memcache I've had issues with previously when it came to JS / ajax calls but that was ages ago by now (very early on in D6). We only have 1 machine currently, if we'd move beyond then the memcache considering over APC (in terms of User bins) would be good to know.

Any feedback on the actual bin allocation that I've linked to?

Recent Memcache seems good

jpamental's picture

Sorry - don't have enough experience to comment about the User bins, but I've been using Memcache successfully in D6 and D7, with a considerable performance boost.

Usual setup includes APC for opcode cache, and memcache for all the cache tables (except the couple they recommend to NOT put in memcache)

Sorry I can't give you any more technical input!

Jason

Jason Pamental
[ @jpamental ]

What do you mean by "internal

Garrett Albright's picture

What do you mean by "internal central authentication system?" If you mean HTTP authentication, of course Nginx supports that.

Also not sure what RHEL has to do with it. I'll eat my sock if there's not an Nginx port for Red Hat. (note: i will not really eat my sock)

Which bins are those and

btopro's picture

Which bins are those and which memcache module are you using since I see two mentioned?

I use the standard 'memcache'

adammalone's picture

I use the standard 'memcache' module on all of my sites with no issue:

  • CentOS release 6.4 (Final)
  • Apache/2.2.15
  • PHP 5.3.3 (FPM)

The configuration options which I use in settings.php and which have worked for me on Ubuntu servers and servers using Nginx are

$conf['cache_backends'][] = './sites/all/modules/memcache/memcache.inc';

$conf['cache_default_class'] = 'MemCacheDrupal';
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';
$conf['lock_inc'] = './sites/all/modules/memcache/memcache-lock.inc';
$conf['memcache_key_prefix'] = 'mysitename_';

$conf['memcache_servers'] = array(
  '127.0.0.1:11211' => 'default',
);
$conf['memcache_bins'] = array(
  'cache' => 'default',
);

Best practices dictate that cache_form is left out of memcache due to it being a transient in memory cache and that information used for building forms in Drupal should not expire/be flushed partway through someone filling in the form. This can lead to the 'this form is outdated' messages sometimes received if a half filled form is returned to way later.

Strictly speaking, the memcache_servers and memcache_bins confs do not to be specified since they're using the defaults. However, I decided to put them in for future expansions. For this configuration, a memcache daemon running on the same server as Apache is used and all bins being placed in memcache are being put into this instance of memcache.

I would agree with the majority of other users in this thread and keep APC as a passive code cache. Set it, and forget it. If there are no APC warnings about needing more memory cropping up then it's probably fine. When using a module like Entity Cache the amount of data to be stored can easily reach >4GB (especially on larger sites) for which a dedicated in memory cache would be ideal (provided you have the memory available).

memcached performance

trainingcity's picture

I recently installed memcached on a multi site server running apc and was amazed at the performance improvement for authenticated users. I assumed the performance improvement would be relatively minor given other caching (views, etc) and the fact that APC was already running. I had avoided memcached thinking it would not be be appropriate for a single server configuration.

I set the memory limit to about two times what I thought I would need and adjusted down from there based on observation.

Here is a nice little memcache/memcached monitoring script. It looks like the built in apc.php script. PS: You might want to add a password to the apc script like the one included here:

http://livebookmark.net/journal/2008/05/21/memcachephp-stats-like-apcphp/

You might also want to run a cache warming script after install to see what type of upper memory limits are appropriate. If you have XML sitemap module running something like this should warm the cache.

Note: This is not generally a good idea on a production site as "there will be blood" on the cpu while the script is running through your entire site contents. It may take several minutes/hour(s) to run. Again, NOT RECOMMENDED if you are on a production server:

Create a file in your /usr/bin directory (Centos) with the following content:

!/bin/bash

Warm the cache.

wget -q http://"@SITE"/sitemap.xml -O - \
| egrep -o "http://"@SITE".com[^<]+" \
| wget -q -i - -O /dev/null

Replace "@SITE" with your site url, ie: example.com

Thank you all for the advice.

btopro's picture

Thank you all for the advice. I'll be investigating memcache for a bigger instance of what I'm currently building as that sounds like the way to go at scale.

I updated the original dropbucket code snippet based on real world performance tweaking of a system I'm running (D7) http://dropbucket.org/node/1046 . While I know many have expressed concerns about using APC user bins, I reset apache nightly to flush opcode and user bins and they do fragment throughout the day but have yet to hinder performance given the traffic I see. Thus far everyone I've showed the sites to with these settings thinks I'm cheating somehow because page loads as user 1 are stupid fast. This is because a lot of the bins live in APC for the heavy lifting ones.

I've also modified the filecache bins to place them in /dev/shm/ which appears to be more responsive then previously. Right now I'm tracking at about 11-13 megs per page load and I'll be reducing the modules in use even further in the near future to try and get that down more.

Going to do a screencast talking through options on drupal planet and reference this thread so thank you all!

Two quick

Dustin@PI's picture

Two quick comments

Theoretically (speed wise): APC > Filecache (/dev/shm) > Memcache Local > Memcache over network > Filecahe (on SSD) ...

Fragmentation in APC can be managed: I've been running my sites "cache_class_cache_bootstrap" and "cache_class_cache" in APC and I have less then 2% fragmentation in my APC cache (Apache hasn't been restarted since November). I have another site (small and read-only) where all my cache bins are in APC and that site only had 4% fragmentation. In both cases I have allotted extra memory to APC to help keep fragmentation low.

Having said that, I am updating my sites to only use APC for a few select cache bins and Memcache for everything else as this seems to be the best config for my use cases.

Filecache (on /dev/shm) should be as fast or faster then Memcache, but Memcache has better management and support, so that's why I am moving my remaining cache bins there (note I have not seen actual bench marks comparing the two, I am basing this on the architectures of the two approaches and some conversations on the Filecache issue queue).

One other reason to not have

Dustin@PI's picture

One other reason to not have cache_bootstrap in memcache, is that be default memcache only supports caching items less then one meg, and some of the cache_bootstrap items can get bigger then that.

There are ways to implement

Spleshka's picture

There are ways to implement cache multipart data with cache chaining. So it is not the reason of not using memcached for big size data (> 1mb).