Posted by PlayfulWolf on June 7, 2010 at 10:21am
Hi,
How is memcache different form APC if both are used as user-object caches, with the same Cacherouter module?
1) APC as opcode + APC as user storage
2) APC as opcode + memcached as user storage
What I see is that in real life I get better results (page refresh times, server load) with case 1) - memory allocation in both cases ~100MB
So is memcache useful at all if using just 1 server? I really doubt it is faster...
What is your experience?
Comments
Good question. APC should be
Good question. APC should be faster since it is in-process and not communicating over TCP. Memcached would be the only option with distributed servers.
I'm using memcached on a single server because the memcached drupal module was more reliable than the cacherouter module several months ago when I was testing them. The cacherouter module seemed a little quirky and didn't clear the cache properly for me when using drush for cron. I see that a new maintainer has been committing to the dev version, so maybe it's better now. I might give cacherouter and APC another go if he releases a new version.
thanx. for me - cacherouter
thanx. for me - cacherouter with APC is working fine. I see "featuritis" when using memcached on single server...
drupal+me: jeweler portfolio
"shared" mode
Also note that, thanks to the "shared" mode setting, using memcached with cacherouter may incur the in-process vs out-of-process extra overhead twice for each request, making it even slower than APC when use on a single host. This is a convenient feature for debugging but should never be used in production (leading to further considerations on how to correctly setup memcache).
The real plus of memcached comes when cache is shared between several web heads, and for data only: opcode caching will still remain in apc/eaccelerator/xcache/zend optimizer/whatever opcode cache you are using anyway.
One benefit of memcache
One benefit of memcache is that it separates the user cache from the opcode cache and helps prevent fragmentation. I saw a lot of cache fragmentation when I had cacherouter using the APC user cache. That makes sense, since there are a lot of small read/write activities on the user cache. Switching to memcache meant the relatively static opcode cache wasn't getting fragmented by a gazillion user cache entries.
It really seems that using
It really seems that using cacherouter with APC for user cache leads to such fragmentation crash, see http://drupal.org/node/766748
On a single server, another option may simply be to use cacher router with file caching for user cache rather than memcache as files may be even cached in memory by the OS. Do you think it is more stable to use file caching for user cache on a single machine or memcache (if we assume that APC crashes as reported)?
strange, I am running APC
strange, I am running APC 3.0.x as user-cache for ~3months on VPS without restart and everything just works.
tested memcache, but its process was spiking for a few minutes to 50-90% of cpu usage almost freezing server when using with authcache. haven't found why. APC works ok.
drupal+me: jeweler portfolio
I've never had any problems
I've never had any problems out of APC either and I ran it for about a year on a site averaging 120,000 hits a day. True I've had to play around with different versions of APC in the past to get one that worked good.
On using file cache, if you have the memory it would be much better to go ahead and throw in memcache and just use that, even if it's only a single instance. Given the slow nature of hard drives, you will see better performance going the memcache route during busy times.
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.
"On using file cache, if you
This statement is generally false. Here's some stats:
http://www.mysqlperformanceblog.com/2006/08/09/cache-performance-compari...
If you are experiencing memcache to have better performance than a file cache this indicates a bug.
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
How much does that test apply
How much does that test apply to Drupal though? That's running off a single cache_get (no sets). It's also caching a straight string. Drupal cache's everything as an object, so when you do file based caching that means everything has to be serialized/unserialized.
I modified his script. Here's running it as is (and memcache has seen significant improvements in the 4 years since Peter did that test):
File Cache Time: 4042885 Gets/Sec: 24734.811897939
Memcached Cache Time: 5820564 Gets/Sec: 17180.465673086
Now make it a simple array (one key=>value) and add in the overhead of serialize/unserialize:
File Cache Time: 4477236 Gets/Sec: 22335.208597447
Memcached Cache Time: 5453238 Gets/Sec: 18337.728886948
They get a lot closer (a difference of 7554 to now 3998).
The same 100,000 loops, but every 100th is also doing a write:
File Cache Time: 6885353 Gets/Sec: 14523.583612924
Memcached Cache Time: 6327119 Gets/Sec: 15804.981698621
So adding in the extra writes you start seeing the performance switch, but they are still relatively close. Actually if you are in a real world Drupal environment, they are probably really close (though that number of writes might be a little high - it depends on the actual site).
Now this test is the only thing running on this server, and it's the same code as Peter posted, just modified to drop out APC and MySQL and to adjust it as stated above. But if you add into it the overhead of Drupal loading (or just stating the files to check for modifications if running APC), plus reading out any static content and all the other server goodness such as MySQL running on it, I'm sure Memcache would do even better.
Also you can get into issues with the file system cache. Linux won't cause apps to swap because of the file system cache. It will drop items from the cache to free up the ram. So it would be interesting to see how the tests pan out when a server is running out of free RAM and having to go to swap. Of course if that's the case, your memcache items are also in swap, so the performance degradation you see would probably be close to the same on each end.
The best recipe in this single server environment would probably be mixing caching mechanisms (something I love in cacherouter). Set up file cache for things that don't change that much (cache_filter, cache_form) and memcache for the items that update a lot. That way you get the best of both worlds.
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.
rad
Very cool to see some numbers with some writes thrown into the mix. This is great information.
You might consider posting these updated numbers on the original blog post at mysqlperformanceblog.com.
--
Dave Hansen-Lange
Director of Technical Strategy, Advomatic.com
Pronouns: he/him/his
Hi, would it be possible to
Hi, would it be possible to see your cacherouter config.
FCGI or mod_php ?
Having APC 3.0 /not/ crash frequently is rather out of the ordinary. Are you using the familiar mod_php or some fastCGI version ?
That is not our experience
That is not our experience. APC 3.x, since 3.0.19, has been very stable with PHP 5.2 (the one that ships with Ubuntu 8.04 LTS). We used to have it with mod_php, and now with FastCGI, and it is very stable.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
3.1.3 has been stable for us
APC 3.1.3 with PHP 5.2.13 and mod_pho on Gentoo has been very stable for us. No issues at all with stability.
I've ran numerous combination
I've ran numerous combination over the years. Only once did I ever have a problem. I'm thinking it was 3.0.18 and all I had to do was go back to 3.0.17 and it cleared right up. That was on a NetBSD server, but I've ran APC on *BSD, Ubuntu and RHEL5 without any problems for awhile now.
HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.
using APC 3.0.19 as opcode
using APC 3.0.19 as opcode and user-cache with prefork Apache, mod_php CentOS 5, 128MB allocated, but rarelly over 80MB used.
tweaked some ttl settings and other not tu use default "0"/"unlimited" settings, I think ttl for 12 hours is ok, havent spotted some king of fragmentation issues, or if they are, they are not slowing Apache.
if speaking not only about Drupal and Cacherouter, had good experiences with eAccelerator, using CentOS systems, when using with memory AND disk setting also with compression levels 1-2 (not 9!!!). the only problem was with garbage collection that sometimes old files, cached on disk were not removed for long time, but in my personal opinion, is is a BETTER product than APC although, you cannot put user-cache in newer versions
drupal+me: jeweler portfolio
Using APC 3.1.3, one issue
Can't figure out why, but APC+Cacherouter as a user cache works for a day or two but then apc's cache_get returns "null" for everything and CPU usage/load goes up.l Using Debian stable + mod_php w/300M allocated for cache. Restarting apache fixes it.
Fragmentation?
When I was using cacherouter to push the Drupal cache tables into APC I was seeing excessive fragmentation in the APC cache. While mine never got to the point where it was producing the null returns you're seeing, I wonder if it's not a similar issue. What does the APC fragmentation look like on your system?
We moved from using the APC user cache to memcache and all our issues went away.
Fragmentation is at 22%
Fragmentation is at 22% currently, will check next time the null values return.
cache fragmentation does stop
cache fragmentation does stop cache usage, see: http://pecl.php.net/bugs/bug.php?id=16911
I'm using memcache only on
I'm using memcache only on big installations when MySQL really runs out of memory and you need to reduce MySQL load. Some additional info and benchmarks here: http://drupalperformanceblog.com/when-you-need-memcache-in-drupal