Testing and Quality Assurance

I just wrote up how I wrote a data-driven webtest of Drupal and I wanted to share my write up.
I copied my full write up, but missing some of the screenshots from http://www.philliprhodes.com/content/testing-drupal-grails-webtest

I was implementing a Drupal site with a custom module that uses a custom mysql database. The client reported problems with a custom module. I tried reproducting the problem, but after trying 20 some combinations out of 1000+ test elements, I wasn't able to reproduce the questionable behavior.

Enter grails with webtest.

Hey all,

Ive been exploring the wonderful Drupal framework for some weeks now, been reading alot about modules, module development and theming. In the last few weeks i have developed some small modules that i want to test using the Simpletest module.
Ive read the documentation and the tutorial to get some sense of how to set up Simpletest for my modules. So i thought i would start by making some functional tests for my custom CCK types. So far i haven't had much luck, ive been wrestling Simpltest for a while now and its not returning the love so far :P

The test bot from testing.dupal.org has fundamentally changed the way that core development happens in Drupal 7. By automatically testing every single patch, and ensuring that Drupal continues to install properly and pass all automated tests, we can ensure that Drupal 7 remains the most stable development release to date, and can save our patch reviewers tons of time that would otherwise be spent manually verifying that patches do not break existing functionality.

However, the test bot urgently needs your help!

Here is a very short module to assist in testing other modules that refer to web services. It is somewhat modeled on what is being done to test openid in version 7, but more generic and simpler. Should I propose that this module get included with simpleTest? Is there other, better ways to do this? Any other feedback would be appreciated. Thanks!

The module I'm working on is a widget module, so its effects can only be seen and tested when at least one content type has been configured with its field.

All this configuration has to be done in this test? In this case, adding a field to a content type and selecting the widget. Any examples, advice, or alternatives for this?

Separate but related: Testing starts each time with a blank slate, but is it possible instead to have the tests run with a copy of a current database? That seems like a good way to help catch bugs that can arise out of different configurations.

Several core tests (contact, comment, poll) have a need to be to adjust the permissions for anonymous users. We can easily adjust permissions for users by calling drupalCreateUser() with different permissions, but we don't have an easy way to do the same thing for anonymous users. As of now, the tests have to add an admin user with the permission 'administer user permissions' and use drupalPost() to adjust the admin/user/permissions page. This implementation is ugly and wasting effort by duplication.

Since we provide the drupalCreateUser() and drupalCreateRole() methods in DrupalWebTestCase, I proposed adding a drupalSetPermissions() that could be easily reused throughout core tests and would be very handy for a lot of contrib module tests. But it needs more reviews.

I just released an install profile for D6 core that sets up a test site for trying out all the combinations of CCK date field configurations: CCK Date Testsite. I've discovered various bugs in Signup's handling of CCK date fields (and have been fleshing out bugs and limitations in DateAPI itself with KarenS's help). I was spending a lot of time just configuring a reasonable test site to try different combinations of CCK date field types (Date, Datestamp, vs. Datetime) and timezone handling (there are 5 different ways date fields can handle timezones). So, I quickly decided the time spent automating the test site would easily save me time in the long run. And, I hope this profile will be useful to other folks working with DateAPI and CCK date fields.