Posted by Swampcritter on March 12, 2009 at 2:43pm
Overview: Drupal Authenticated User Scalability
Description: Currently, Drupal has many modules available that help for caching with unauthenticated users visiting a website (i.e. APC, Cache Router, Memcached API, Boost, Throttle, Drupal Cache, etc). However, when it comes to authenticated users, there aren't really any modules available. About the only solution -- to present date -- would be to use something like Squid reverse proxying or something similar. That's all nice and everything, but that really takes away from Drupal being self-sufficient and requires a bit more tech-knowledge than most developers or site administrators want to deal with.
Mentors:
- Jonah Ellison :: Developer of Authcache
- Michael Worsham :: Recommendation of Project, Platform for Testing
- OPEN :: Someone else with Core and/or Cache development
Difficulty: Most likely Medium to Hard
Comments
nice idea
i like this idea, and i'd be prepared to mentor for it.
there have been a couple of attempts at this, and it would be great to get more work done on improving performance for logged in users.
Looks like something in progress
http://groups.drupal.org/node/19823 - Authenticated User Page Caching (Authcache)
Moved to the official ideas list.
Even if that module is a good first step, I'm sure there will be a lot more work (and benchmarking) to be done (not sure if it will be enough for an entire SoC project- we'll have to see).
--
Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology
Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology
I would love to see who the
I would love to see who the possible mentors for this would be. I'm pretty sure this would require folks that normally work on core issues. (I thought SOC tried to avoid working on core stuff). Would this be implemented entirely in contrib?
Where is the beef?
This is a problem Drupal has. How do you intend to solve it? What would be a "pass" for this project? What code do you want to see?
Goals & Issues
The Authenticated User Page Caching (Authcache) module (developed by Jonah Ellison) [ http://groups.drupal.org/node/19823 ] is a step in the right direction, however it relies on using the CacheRouter as a base and quite possibly (as per explanation below) would not allow the Authcache and CacheRouter modules to work independently.
Issue #1: Steve Rude (the original developer of CacheRouter) noted that the Authcache cannot work at the same time the CacheRouter module is running... "cause the cache router module itself does very little. all the logic is in cacherouter.inc." [ http://groups.drupal.org/node/19823#comment-69818 ]
Issue #2: Currently under D5/D6 and above, CacheRouter allows the use of Memcached, APC and other types of caching capabilities. If Authcache disables this capability of running concurrently with the CacheRouter, then large Drupal sites will have problems handling high get and post requests. Having to rely on Drupal caching alone is not really an option and will definitely lead to performance and scalability issues.
Issue #3: The underlying problem is that authenticated users are never 'cached', thus will result in higher memory, CPU and disk I/O contention problems upon MySQL of having to retrieve images and data from the database environment.
Since MySQL Clustering is not a supported option (at this time) due to the NDBCLUSTER engine issues -- about the only option, to date, is to either use MySQL-Proxy [alpha based application -- http://forge.mysql.com/wiki/MySQL_Proxy ] or to use a code/patch bundle (aka 'unsupported hack' developed by several Drupal volunteers) that modifies the Drupal core environment for allowing the MySQL database to use split reads and writes for queries and insert/updates [ http://sourceforge.net/projects/msrwd ].
Goal: A 'pass' for this project would most likely involve a recode of the Authcache module not to use the CacheRouter as a base -or- to find a way to make the Authcache to work independently with the CacheRouter base code, thus allowing Memcached and/or APC to be utilized as well.
Integration with Cache
Integration with Cache Router is something I planned to address this weekend--it would actually be a rather straightforward process and wouldn't require any major recoding. I'll be opening an issue in the Cache Router project page for Steve Rude to look at and get the ball rolling, since he'll need to add some minor modifications to the Cache Router module.
I think there is some confusion with the difference between Cache Router & Authcache. Right now, Authcache simply uses a fork of Cache Router, which means it is built on top of Cache Router. This means it has the same functionality as Cache Router. It doesn't disable anything and supports memcached/APC exactly the same way as Cache Router.
http://httpremix.com
Mentoring/Authcache
If this involves improving the Authcache module, I'll mentor on this. Currently, I see the following work:
http://httpremix.com
Note
Just a note for consideration. If you don't know, SwampCritter is a professional sysadmin, and has inherited support for some of the larger Drupal sites in the newspaper industry (including the old SavannahNow.com, which has a pre-Boost version of static page caching).
He's looking to scale sites like http://jacksonville.com and http://cjonline.com, which means that we should really take advantage of the opportunity to do some scalability testing in his sandbox.
So consider that my endorsement of the goals of this project, though I agree with chx that we need a strong definition of what a 'pass' looks like. Perhaps benchmarks would help.
E.g. Improve page load times by 75% for authenticated users on test site X.
--
http://ken.therickards.com/
--
http://ken.therickards.com/