Last night I (finally) updated my (ultra-simple) personal website to 6.0, and while I was at it I decided to try out some caching ideas that had been brewing in the back of my mind for a while.
Though all caching strategies have their pitfalls and drawbacks, the idea I wanted to try -- combining Boost and Memcache -- seems like a real live-wire. However, if it can be made to work, I think it might offer some significant advantages.
The origin for this idea comes from my experience with both modules. Boost's static file caching is the heavyweight champion for anonymous traffic as far as I've seen. It's essentially the poor-man's reverse proxy, preventing the http requests from even being handled by Drupal. This gives the best results I've seen yet in terms of handling big spikes (e.g. a slashdotting).
On the other hand, I've generally experienced Memcache as providing a tangible boost for logged-in users. My thinking is that because it places the load for cached application data (e.g. variables, views, menu tree, etc) into memory rather than on the SQL back-end, the data can be loaded more quickly, and this makes for snappier page execution times.
Running both of these systems seems like a workable idea. So far my site is running just fine, but again it's a very very simple use-case. While there's some redundant caching of pages -- e.g. on a fresh anon pageload, the page will end up cached in the database, memcached, and on the filesystem via boost -- that doesn't seem like a huge drawback. I'm curious if anyone has any experience here, thoughts to add, or holes to poke in this theory.
Below are some benchmarking numbers. I am laying siege to /node with 25 concurrent users in benchmarking mode for 1 minute, a pretty righteous surge of traffic (equivalent to close to a quarter-million visits in an hour). I am also watching top to observe server-load. This is all Drupal 6 on a mid-range VPS slice w/1gb of ram and APC enabled.
Initial System w/Regular Drupal Page Cache
Load under siege: 2.24
Siege result detail:
Transactions: 4135 hits
Availability: 100.00 %
Elapsed time: 59.52 secs
Data transferred: 119.89 MB
Response time: 0.35 secs
Transaction rate: 69.47 trans/sec
Throughput: 2.01 MB/sec
Concurrency: 24.58
Successful transactions: 4135
Failed transactions: 0
Longest transaction: 10.07
Shortest transaction: 0.12Memcache Enabled (one server w/100Mb of cache; simple simple)
Load: 2.91
Siege result detail:
Transactions: 3477 hits
Availability: 100.00 %
Elapsed time: 60.42 secs
Data transferred: 100.81 MB
Response time: 0.43 secs
Transaction rate: 57.55 trans/sec
Throughput: 1.67 MB/sec
Concurrency: 24.74
Successful transactions: 3477
Failed transactions: 0
Longest transaction: 28.22
Shortest transaction: 0.11As you can see, this is actually slightly worse. I think this is because of the extra overhead of running memcached and the fact that since my site is so simple and the requests are all for the same page, Drupal's normal/basic page-cache is actually more efficient w/its results already loading into the mysql query cache.
However, in a more complex configuration, this isn't often the case. Also, as I mentioned, memcached provides a speedup for logged-in users since Drupal doesn't need to hit the SQL to load cached application data like the menu tree, variables, block-cache, etc.
Boost
Load: 0.15
Siege result detail:
Transactions: 4527 hits
Availability: 100.00 %
Elapsed time: 60.44 secs
Data transferred: 131.67 MB
Response time: 0.32 secs
Transaction rate: 74.90 trans/sec
Throughput: 2.18 MB/sec
Concurrency: 23.91
Successful transactions: 4527
Failed transactions: 0
Longest transaction: 11.05
Shortest transaction: 0.09As you can see, Boost is pretty unbeatable when it comes to logged-out page caching. However, it doesn't help at all for logged-in users. I think by combining the two, we can get the best of both worlds.
Comments
Isn't that what CacheRouter is supposed to do?
Cache Router could do this, no?
Well, not really
Since Boost operates at the Apache layer of the stack, it's not really controllable by CacheRouter (or anything else in Drupal, for that matter). If there were some way for Drupal to manage the boost portion of the .htaccess file that would be different, but I'm not sure that's possible...
http://www.chapterthree.com | http://www.outlandishjosh.com
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
Some comments
With boost, I have seen good performance, but there were two issues at the time (that was Drupal 5.x):
As for memcache, I always use it without the dual db/memcache backend, and go for the memcache only backend. This takes out a lot of overhead of disk I/O and MySQL CPU processing, adn gives the best possible performance.
Try that and rerun the benchmarks and see if it is better. I have consistently seen this to be better than Drupal's normal page cache alone, and your reslts have memcache slower than page cache which should not be the case.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Hmmm
It turns out that I was using the memcache.inc (so my note about db overhead was wrong). When I switched to memcache.db.inc the performance was indeed worse. Load peaked at 13.46:
Transactions: 1401 hitsAvailability: 100.00 %
Elapsed time: 59.74 secs
Data transferred: 40.54 MB
Response time: 1.05 secs
Transaction rate: 23.45 trans/sec
Throughput: 0.68 MB/sec
Concurrency: 24.71
Successful transactions: 1398
Failed transactions: 0
Longest transaction: 15.09
Shortest transaction: 0.21
As I said before, the specifics of my setup (including the middling VPS quality) might account for some bottlenecking for memcached. It's quite possible that this could be greatly improved with some tuning. These benchmarks should not be considered very authoritative. ;)
There are indeed some issues still with Boost, but it's improving as it's getting more attention. I hope that someday soon it can be considered a go-to solution for sites that need to withstand the slashdot effect.
http://www.chapterthree.com | http://www.outlandishjosh.com
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
Indeed, not authoritative
I just re-ran all my benchmarks and they were all performing much worse (even though my codebase was the same). In this round, memcache.inc beat out the simple page cache ever so slightly.
I think this is my VPS provider and other activity on the physical server.
I have been wanting to set up an amazon EC2 instance for doing this kind of benchmarking. It would be neat to try that out as a standard "server unit" and to use a more useful setup (e.g. a vanilla site populated by devel generate) as opposed to my own blog for testing. :P
http://www.chapterthree.com | http://www.outlandishjosh.com
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
Bare iron
Strange.
I prefer running my benchmarks on a bare iron relatively quiescent server that is local, rather than something up "in the cloud" where you don't know what the physical server is doing.
Even a relatively cheap AMD Athlon 64 X2 with 1 or 2 GB will do. Of course dual quad Xeons are nicer, but ... ;-)
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
hehe
I keep meaning to re-purpose an old server on the local network, but it keeps falling to the bottom of my TODO list. ;)
http://www.chapterthree.com | http://www.outlandishjosh.com
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
Dangling symlinks?
Can you elaborate on the "dangling symlinks" issue? Does this refer to deleted content? We don't have much of that.
We're not using Boost (not yet anyway), but we are using Squid/Squirm to cache static files out in front of multiple Drupal servers, with Memcached handling Drupal caches, all on 5.x.
Defining dangling ...
Boost caches Drupal pages in files. If there is an alias to the page, then it does not cache the page twice, but creates a *NIX symbolic link that points to the original file.
For some reason, the original page can be deleted from boost cache, probably when it expired, but the symlink to it is left in the file system pointing to a deleted file.
If you don't use boost, then none of that applies. Memcache and Squid should be a better solution for larger sites.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc..
Personal blog: Baheyeldin.com.
Patch solving Boost's symlink problems
There is a bjaspan's patch for Boost -Remove symlink creation. Let each path have own file- to stop using symlinks.
This also solves Windows compatibility, and i18n problems for multilingual sites. That patch is recommended on Support locale and multilanguage sites, and on Improving Drupal's Performance with the Boost Module for the UN's Millennium Campaign.
Boost's symlink problems fixed
Today (May 2, 2009), Boost 6.x-1.0-alpha3 has been released. From the release notes: "Symlinks are no longer created/used. Windows should now work! Better i18n support as well."
Apart from this, there is an advisable, easy .htaccess modification to preserve browser cache, by moving a few .htaccess lines to the cache folder. I think this modification is not officially included yet, and needs to be done by hand for now.
stale pages with boost
I ran into this issue once before. The problem is when you first install boost, any logged in users don't have the cookie which it uses to differentiate - so to avoid serving stale pages, you can TRUNCATE sessions, and once people have logged in again, everything will go smoothly - at least in terms of stale pages. But there's not an obvious solution to this.
Other evidence this works
Looks like others have also been doing this successfully:
http://drupal.org/node/179833
I feel a blog post to Drupal Planet coming on... ;)
http://www.chapterthree.com | http://www.outlandishjosh.com
https://pantheon.io | http://www.chapterthree.com | https://www.outlandishjosh.com
Helpers in CacheRouter?
Think that CacheRouter might be the right place for some "boost helpers" to live? I can imagine some path-based caching or other stuff like that which could work really well.
we do this...
at http://www.nanowrimo.org and its sister sites (running drupal 5)
we've made some mods to boost to handle the symlink issue, and caching for pages with query strings (i think there are code snippets for both in the boost queue)
we're also running a heavily modded advcache.
there's been a lot of talk about using memcache and/or cache router to handle page caching without hitting the database or initializing the full bootstrap in php, and if that works i suspect it's an adequate/superior replacement to boost. however, i haven't had the time to become smart enough to follow that conversation yet.
fastpath file system cache
There is Fastpath fs_cache which is a filesystem based cache. I think it avoids calls to the DB but still executes a fair amount of PHP which makes it less desirable than straight up Boost.
Great work Josh - now to redo it with an isolated system ;)
--
Growing Venture Solutions | Drupal Dashboard | Learn more about Drupal - buy a Drupal Book
knaddison blog | Morris Animal Foundation
Patching Boost...
Firebus, thanks for your interesting article National Novel Writing Month and Drupal Scalability, which for instance includes solutions to improve Boost, and other practical advice.
lighttpd
I would also redirect all static boost page and image requests through lighttpd. I just rerouted them to lighttpd running behind Apache through this technique:
http://www.linux.com/feature/51673
If you are using imagecache module there are some additional things to do.
I haven't been able to get any module that requires a serialization patch to take on my 5.0 installation so after I upgrade to drupal six I was planning on trying a combination of cacherouter and boost.
I can attest that
I can attest that cacherouter + boost works well. I use boost for serving our homepage for all 5 languages we support with a small hack to .htaccess and adding some links manually to cache/SITE/$LANG.html to point to the appropriate node.
I will have a new version of Cache Router available very soon that has session support as well. So you should be able to use it with the patches available at http://www.tag1consulting.com/ as well.
Slantview Media http://www.slantviewmedia.com/ | Blog http://www.slantview.com/
Have you done any basic
Have you done any basic stack tuning? opcode cache?
He's got APC
He's already got APC running.
In performancecomparisons, you should account for multiple PHP instances (i.e web servers at the end of the line) APC will offer the maximum benefit when requests are handled repeatedly by the same PHP instance; it is a local cache. I've seen described an interesting setup, where they managed to avoid moving sessions from a server to the other; they used only APC. Memcached works at the database layer, so it will accelerate all requests, whatever server gets hit.
This is an old thread, but i
This is an old thread, but i hope someone can point me in the right direction. I am using drupal 6.26 with Boost, APC and memcache. APC and memcache are installed and recognized according to phpinfo. Here is my php.ini:
extension=memcache.somemcache.hash_strategy="consistent"
memcache.default_port="11211"
extension=apc.so
extension=uploadprogress.so
apc.shm_segments=1
apc.shm_size=128
apc.stat=1
apc.ttl=0
apc.user_ttl = 0
apc.gc_ttl = 3600
apc.cache_by_default = 1
apc.max_file_size = 1M
apc.rfc1867 = 1
apc.rfc1867_prefix = "upload_"
apc.rfc1867_name = "APC_UPLOAD_PROGRESS"
apc.rfc1867_freq = 0
apc.rfc1867_ttl = 3600
and here is my settings config:
$conf = array('cache_inc' => './sites/all/modules/memcache/memcache.inc',
'session_inc' => './sites/all/modules/memcache/memcache-session.inc',
'memcache_servers' => array(
'127.0.0.1:11211' => 'default',
'127.0.0.1:11212' => 'block',
'127.0.0.1:11213' => 'content',
'127.0.0.1:11214' => 'filter',
'127.0.0.1:11215' => 'form',
'127.0.0.1:11216' => 'menu',
'127.0.0.1:11217' => 'page',
'127.0.0.1:11218' => 'update',
'127.0.0.1:11219' => 'views',
'127.0.0.1:11221' => 'session',
'127.0.0.1:11222' => 'users'
),
'memcache_bins' => array(
'cache' => 'default',
'cache_block' => 'block',
'cache_content' => 'content',
'cache_filter' => 'filter',
'cache_form' => 'form',
'cache_menu' => 'menu',
'cache_page' => 'page',
'cache_update' => 'update',
'cache_views' => 'views',
'session' => 'session',
'users' => 'users'
),
);
I am getting virtually no noticeable performance boost for authenticated users. When looking at memcache reports, im getting 100% misses on page bin, 1% misses on users and session bins. I can provide more stats if needed. Please point me in the right direction.
Why are you getting no hits / all misses ?
I think the problem is that you are missing the cache all the time.
What is the size of your cache bins ? Maybe they are tiny ?
Perhaps caching is disabled in the performance settings ?
--Rob
I had the cache bins open to
I had the cache bins open to 128mb at one point just to be sure that wasn't the issue. I've also tried with core caching both enabled and disabled with the same result.