Benchmarking Drupal

Are performance optimisations still going into D6?

robertDouglass — Mon, 28 Jan 2008 15:07:59 +0000

I'm assuming it's too late, but if not here's an easy performance win for D6:

The type column of the {system} table is currently a varchar(255) field. Yet in 99% of all Drupal installations it stores either the text 'module' or the text 'theme' and nothing else. This is already a strong case for making it an int column and defining constants for module and theme. Then, on every page, there is this query:
SELECT filename FROM system WHERE name = 'user' AND type = 'module';

This query always takes over 1 ms to run making it not a criminally slow query, but relatively slow, nonetheless. This is because the list of indexes available doesn't help the query much

By shortening the {system}.type column to 32 chars and adding an index we reduce the query by around 60%... and this query runs on every page load.

Postgresql

Are you storing your Drupal sites or configs in a version control system? If so, which?

ccharlton — Mon, 17 Sep 2007 21:12:02 +0000

Workflow

Presentation: Drupal performance tuning and optimization

kbahey — Mon, 03 Sep 2007 08:45:26 +0000

OpenCraft hosted a 3.5 hour seminar that I presented on Drupal performance tuning and optimization. You can find the slides from the presentation there, which can be useful to some of you.

High performance

Drupal memory consumption by core and modules

kbahey — Mon, 03 Sep 2007 08:45:00 +0000

I create a 5.2 and 6.x patch for measuring how much memory a module takes, and also how much the whole Drupal page load takes.

The results are published here: Measuring memory consumption by Drupal bootstrap and modules.

Short version: in Drupal 6, modules are better off than in 5.2, but the bootstrap is a tad larger.

If anyone is interested in replicating the results with more realistic install profiles, please do so and publish your results/findings/recommendations.

Also, I am looking for opinions on incorporating this memory measurement in core and devel modules. If you think this is a good idea, please say so.

High performance

Convenient SQL transactions with PressFlow Transaction

David Strauss — Sat, 31 Mar 2007 22:03:46 +0000

I've released an in-development version of PressFlow Transaction for developers interested in convenient encapsulation of SQL transactions. The key features are intelligent use of scope for COMMITs and ROLLBACKs as well as safe, intelligent nesting of transactions to get exception-like semantics.

Usage details are on the project page. Requires PHP 5.

(I posted this to the High-Performance group because encapsulating updates in transactions can dramatically improve performance.)

High performance

SoC 2007 proposal - Drupal Automated Staging Kit

Allister Beharry — Fri, 23 Mar 2007 17:33:28 +0000

Hello all,
Sorry for cross-posting; but I think people working on different aspects of Drupal may have interest in this idea.

I've submitted a SoC 2007 Drupal proposal on automating the whole process of creating a complete Drupal site with a LAMP stack in a self-contained virtual machine image. A formatted PDF version is here:
http://www.abeharry.info/SoC2007_DrupalAST_FullProposal.pdf
Here is the abstract:

The Drupal automated staging toolkit is a proposed set of code libraries, file schemas and parsers, and code generators, for
automatically creating a Drupal site with specific module code versions, sample users and data, and a specific LAMP stack
configuration for hosting the Drupal site. The toolkit also has the ability to stage this generated site on an existing physical server
location, and also as a self-contained virtual machine consisting of a minimal Linux environment, required LAMP software, and the Drupal
site.

The automated staging toolkit is designed to be part of an automated unit- and regression testing environment, by providing testers with a
simple, fast way to automatically generate a complete Drupal site running specific code versions, and using specific LAMP server
configurations. It is also intended for use as part of a performance and scalability testing environment by providing the ability to
rapidly build and then benchmark the effects of different application, web and database server configurations on Drupal site performance and
scalability.

This toolkit will be used in the following way:
1. The tester creates or reuses an XML(or other structured) file using
a schema describing the Drupal site code-tree, including modules
installed/enabled/disabled, and the versions of each module to be
used.
2. The tester creates or reuses an XML file using a schema describing
the LAMP stack web server, PHP/application server, and
database server configuration; e.g Apache vs. Lighttpd, mod_php
vs.FastCGI, choice of op-code cache, MySQL vs. PostgreSQL, and so on.
3. The tester creates or reuses an XML file using a schema describing
the sample users and content data the site will contain.
4. The parsers take each file and generate scripts in a lightweight
language (Python or Ruby or PHP-CLI.) These scripts, when executed,
use functions in the code libraries to download Drupal modules,
generate database scripts and datasets, and write server configuration
files.
5. Given a physical server target location, the Drupal modules,
database scripts and server configuration files are deployed to the
designated server location, to produce a new, ready-to-test Drupal
site.

Time and resource permitting, a builder in the toolkit will also use the generated Drupal site and servers' configuration as input to build
a self-contained virtual machine image in Xen, VMWare, or potentially the Amazon EC AMI format. This virtual image can also be be used in
testing environments, including advanced performance testing scenarios such as evaluating clustering, distributed database topologies, and
alternative storage and computing models like Amazon S3 and Elastic Cloud. The ability to rapidly generate self-contained Drupal virtual machine images will also be extremely valuable to Drupal consultants and solution providers for marketing, prototyping and demonstrating
Drupal solutions to potential clients, and large organizations and ASPs like CivicSpace looking to take advantage of the massive benefits
of virtualization technology from VMware, Xensource, Amazon, and the like.

Allister.

SoC 2007

New caching module for complex and high-load sites

David Strauss — Thu, 01 Feb 2007 07:15:59 +0000

I've just released two modules for developers and maintainers of performance-intensive sites. The first module provides a new, self-tuning, preemptive cache API for modules. It's called PressFlow Preempt. The second module (named PressFlow Preempt Panels) builds on the first by seamlessly offering cached versions of Panels.

High performance

Building a Drupal benchmarking test suite

Gerhard Killesreiter@drupal.org — Mon, 22 Jan 2007 02:16:26 +0000

Hi there, I have some interest in building a Drupal becnhmarking suite that can be used for testing the impact of newly proposed patches on our beloved content managemanet framework.

I already have a start of a setup (which needs to be improved a lot). Currently, the setup is a rather simple Drupal site with content generated by the script collection that comes with devel.module. I have commented out the "log out" link and then let five registered users browse the site. The registered users are implemented through wget. This has some advantages and some disadvantages. Advantage is that the users more or less follow the same path when traversing the site. At least this was an advantage when I wanted to study the influence of several patches on table locking. For other tests it might not be terribly useful.

RobRoy has shown me how to do some comment posting, so I am likely to add this somehow.

The question is how the whole think should be organized. Currently, it is a standalone shell-script which calls the wgets with session info attached. What I would like to see is a bacnhmark.php script which lets a user specify the number of users to be used, the rate of traversal, the rate of comment posting, etc, and then starts the processes in the background. Ideally this would happen on a separate server. benchmark.php would then collect the results and do some statistical analysis. The test results would then optionally be specified in a database of test results, maybe here on drupal.org.

What I think will be most challenging is not to write the code, but to define useful test setups. We'd need to analyse how users really use a site and would probably need (anonymized) data from accesslogs.

Also, we'd want to check how crawlers access a site, whch can also be destilled from accesslogs.

Anybody interested in collaborating on this?

Please add useful ideas in the comments.

Cheers,
Gerhard

Benchmarking Drupal

Benchmarking drupal.org upgrade

ChrisKennedy — Thu, 21 Dec 2006 01:16:47 +0000

One thing I would be really interested in seeing is a benchmark suite run on the current drupal.org installation, and then again once it's upgraded to 5.0. Is this something that could be done or is being planned? It would be an underestimate of performance gains to the extent that certain performance patches are applied directly to d.o when fixed in core.

Performance upgrades from the CHANGELOG include:

Improved session handling: reduces database overhead.
Improved access checking: reduces database overhead.
Made it possible to do memcached based session management.
Omit sidebars when serving a '404 - Page not found': saves CPU cycles and bandwidth.
Added an 'aggressive' caching policy.
Added a CSS aggregator and compressor (up to 40% faster page loads).

There have also been numerous micro-level patches that optimize specific SQL queries or otherwise improve performance in certain situations. And a few other good performance fixes are still in the queue and may or may not make it into the 5.0 release.

Benchmarking Drupal

Graphing out watts used with Drupal

bertboerland — Fri, 25 Aug 2006 13:48:31 +0000

Inspired by Dries posting on the powersavings one can get by having a system run on low CPU usage:

Say we would make a test with 1k nodes in Drupal and Joomla? and run JMeter with the same parameters on it against both. Both would be optimzed for caching.

It would be nice to make some MRTG/RRD stats on it with using the MIBS of Watts used and CPU's cycles burned. Then we can show that Drupal will burn less cycles and is in fact environmental friendlier then Joomla?.

Save the environment, eat a beaver and use Drupal :-)

Benchmarking Drupal

Some initial documentation

webchick — Fri, 18 Aug 2006 17:52:07 +0000

I made an attempt at starting some documentation around benchmarking code:

http://drupal.org/node/79237

Reviews/Additions would be appreciated. :)

Benchmarking Drupal

excellent idea - some starting points

greggles — Mon, 14 Aug 2006 15:40:59 +0000

I've posted on standard benchmarking information before (with no responses...) in the performance and scalability forum which is a pretty good source of information in general.

Benchmarking Drupal