Building a Drupal benchmarking test suite

Hi there, I have some interest in building a Drupal becnhmarking suite that can be used for testing the impact of newly proposed patches on our beloved content managemanet framework.

I already have a start of a setup (which needs to be improved a lot). Currently, the setup is a rather simple Drupal site with content generated by the script collection that comes with devel.module. I have commented out the "log out" link and then let five registered users browse the site. The registered users are implemented through wget. This has some advantages and some disadvantages. Advantage is that the users more or less follow the same path when traversing the site. At least this was an advantage when I wanted to study the influence of several patches on table locking. For other tests it might not be terribly useful.

RobRoy has shown me how to do some comment posting, so I am likely to add this somehow.

The question is how the whole think should be organized. Currently, it is a standalone shell-script which calls the wgets with session info attached. What I would like to see is a bacnhmark.php script which lets a user specify the number of users to be used, the rate of traversal, the rate of comment posting, etc, and then starts the processes in the background. Ideally this would happen on a separate server. benchmark.php would then collect the results and do some statistical analysis. The test results would then optionally be specified in a database of test results, maybe here on drupal.org.

What I think will be most challenging is not to write the code, but to define useful test setups. We'd need to analyse how users really use a site and would probably need (anonymized) data from accesslogs.

Also, we'd want to check how crawlers access a site, whch can also be destilled from accesslogs.

Anybody interested in collaborating on this?

Please add useful ideas in the comments.

Cheers,
Gerhard