Testing and Quality Assurance

Functional SimpleTest Status - List of all tests

boombatower@drupal.org — Wed, 05 Mar 2008 16:48:43 +0000

Purpose

This list serves as on overview of the current functional testing status. This will help keep track of who is writing tests and what tests are left to be written.

How you can help

We need people to help write tests. If you can program then feel free to write some tests. If you are not into coding you can help with running simpletest_automator and recording tests.

Maintainer

Jimmy Berry (boombatower) is maintaining this list. If you begin work on a test please contact me and I will be happy to update the list. If there is any information that is incorrect please notify me.

If you have any questions please contact me.

Status

We currently have 38 tests taken out of 43 for a test coverage of 88%. This will change when the misnamed tests are merged.

List

Module name	Status	Issue	Author(s)	Misnamed
aggregator	Completed	http://drupal.org/node/231481	csevb10, kscheirer, poetro
block	Completed	http://drupal.org/node/231313	boombatower, KentBye
blog	Completed	http://drupal.org/node/240644	solotandem
blogapi	Completed	http://drupal.org/node/209146	boombatower
book	Completed	http://drupal.org/node/207192	boombatower
color
comment	Completed	http://drupal.org/node/212261	boombatower
contact	Completed	http://drupal.org/node/232665	boombatower
dblog	Completed	http://drupal.org/node/241467	solotandem
filter	Completed	http://drupal.org/node/210666	boombatower
forum	Completed	http://drupal.org/node/244602	solotandem
help	Completed	http://drupal.org/node/242475	solotandem
locale	Completed	http://drupal.org/node/206342	chx
menu	Completed	http://drupal.org/node/245226	solotandem
node
node (node_revisions)	Completed	http://drupal.org/node/201138	cwgordon7	x
node (node_teaser)	Completed	http://drupal.org/node/210184	bevan	x
node (page_creation)	Completed	http://drupal.org/node/212123	beeradb	x
node (page_view)	Completed			x
node (story_edit)	Completed	http://drupal.org/node/196700	corniI	x
node (story_preview)	Completed	http://drupal.org/node/196700	corniI	x
openid
path	Completed	http://drupal.org/node/207196	boombatower
php	Completed	http://drupal.org/node/243813	boombatower
poll	Completed	http://drupal.org/node/205014	chx
profile	Completed
search	Completed
search (search_match)	Completed	http://drupal.org/node/205631	steven	x
statistics	Stub + initial test committed	http://drupal.org/node/248436	vonster
syslog	Completed	http://drupal.org/node/243861	boombatower
system	Completed	http://drupal.org/node/243367	boombatower
system (modules_system)	Completed	http://drupal.org/node/201134	Smartys	x
taxonomy	Completed
taxonomy (taxonomy.module)	Completed			x
throttle
tracker	Pending Review	http://drupal.org/node/239455	dmitrig01
translation	Completed	http://drupal.org/node/207385	boombatower
trigger
trigger (content_actions)	Completed	http://drupal.org/node/201139	cwgordon7	x
update	In Progress	http://drupal.org/node/253501	dmitrig01
upload	Completed	http://drupal.org/node/234642	boombatower
upload (upload_tests)	Completed			x
user	Completed	http://drupal.org/node/203780	chx

Test State

Below is listed each of the written tests and their current status. This list needs to move towards all passes and reviewed.

Module name	Written	Merged	Passes	Reviewed	Issue
aggregator	X		X	X	http://drupal.org/node/239197
block	X		X	X	http://drupal.org/node/240362
blog	X		X	X	http://drupal.org/node/243966
blogapi	X		X	X	http://drupal.org/node/245955
book	X		X	X	http://drupal.org/node/240378
color
comment	X		X	X	http://drupal.org/node/240599
contact	X		X	X	http://drupal.org/node/240278
dblog	X
filter	X		X	X	http://drupal.org/node/240635
forum	X		X
help	X		X
locale	X		X
menu	X		X
node	X	X
openid
path	X
php	X		X	X	http://drupal.org/node/243969
poll	X		X
profile	X
search	X	X	X	X	http://drupal.org/node/244603
statistics
syslog	X		X
system	X	X	X
taxonomy	X	X	X	X	http://drupal.org/node/245887
throttle
tracker
translation	X		X	X	http://drupal.org/node/244704
trigger	X	X
update
upload	X	X	X
user	X			X	http://drupal.org/node/244763

Testing and Quality Assurance

Rename the "Unit Testing" group to "Testing"

Damien Tournoud — Wed, 02 Jul 2008 15:19:25 +0000

Today the Drupal Testing framework is many an UI ("functional") testing. Should we rename this group to simply "Testing"?

Damien Tournoud
http://drupalfr.org

Testing and Quality Assurance

SimpleTest module's faults at first sight

internal@drupal.org — Mon, 16 Jun 2008 04:31:34 +0000

1. Too heavy to write simple test

if I changed code in blog_link to remove author's blog link, how can I write a small code to ensure it? logic as

testnode = new node;
node->type = blog;
testlink = blog_link('node', testnode);
assertNotContains('tom's blog', testlink);

2. Not IDE friendly, even for simpletest series.

Using simpletest eclipse plugin, you can't run test file because of the .test extention, have to rename to .php. After renaming, it still can be run because of complex file including.

Any thoughts?

Testing and Quality Assurance

Tests that fail by design

catch — Fri, 06 Jun 2008 12:14:03 +0000

We're down to under 60 core test failures, and there's at least four RTBC patches in the queue which will reduce that further, meaning we'll be down to 5-6 unresolved issues dealing with core test patch failures pretty soon.

At the moment, there's only one test that fails by design - for the url filter. However that core bug is likely to get fixed before all tests are working again, so it's unlikely to impact on much.

However, once we do have all tests working, and some coverage gaps filled, then there's bound to be some tests we'll want to write that fail by design - when just running all tests, it's almost impossible to tell that this is the case though, so I'm wondering how best to deal with it.

We could put a note in code comments, or maybe in the assertion response text, link to an issue nid for the bug (that could be left in as documentation for the test even when it's fixed). Either way I'd be interested to see what people think about this, and different ways to handle it. I'm also not sure how it'll play with the DrupalTestBed which I guess assumes 100% pass rate?

Testing and Quality Assurance

Known test failures

catch — Wed, 04 Jun 2008 10:43:54 +0000

If you're looking for current status, look at the most recent comment

Since we've still got core test failures, and no automated patch testing, I thought it'd be worth doing a snapshot of tests that pass week by week (or more frequently if there's lots of commits). This allows us to keep a historical overview of where bugs have been introduced into either core code, simpletest.module or tests themselves. It should also hopefully help to reduce duplicate issues about broken tests (since a test failure can be down to any one of those conditions).

Testing and Quality Assurance

XML output for simpletest requests

dropcube@drupal.org — Tue, 03 Jun 2008 17:49:41 +0000

I have created a feature request at http://drupal.org/node/266220

Feedback please.

Testing and Quality Assurance

Getting Involved in SimpleTest

boombatower@drupal.org — Wed, 21 May 2008 21:35:13 +0000

Now that SimpleTest is apart of Drupal core issues related to the 7.x development have been moved to the core issue queue. If you would like to be involved I have included two links which will filter the issues to only display SimpleTest related issue. You can obviously used the advanced search feature to narrow your results to the SimpleTest related components instead of using the links.

7.x Related Issues

SimpleTest module:
Component: simpletest.module
http://drupal.org/project/issues?projects=3060&components=simpletest.mod...

Tests
Component: tests
http://drupal.org/project/issues?projects=3060&components=tests&states=1...

5.x and 6.x

All issues
http://drupal.org/project/issues/simpletest

Testing and Quality Assurance

Undefined index exceptions

angeliti@drupal.org — Thu, 15 May 2008 09:55:18 +0000

Hello everyone!

I keep getting 'Undefined index' exceptions in my tests:

Unexpected PHP error [Undefined index: weight] severity [E_NOTICE] in [/Applications/MAMP/htdocs/drupal-5.7/modules/taxonomy/taxonomy.module line 502]

As you all know, they occur in this kind of situations:

$example = array('name' => 'example');
if ($example['weight']) {
  ...
}

One option to avoid this is to check if the variable is set:

if (isset($example['weight'])) {
  ...
}

However, this is not done in the Drupal core code, so we will keep getting the exceptions when using functions like 'save_node' or 'taxonomy_save_term'.

The other option is to define EVERY index in our arrays, and just leave empty the ones we are not going to use. This is annoying and makes the code less legible.

What is the right way to proceed to avoid these exceptions?

Thank you!
Ángel.-

Testing and Quality Assurance

Correct simpletest syntax for adding tags to a story/page form?

angeliti@drupal.org — Wed, 14 May 2008 07:24:52 +0000

Hello everyone!

I'm having problems when trying to add tags to the story/page creation form using simpletest.

I have a vocabulary which uses FREE TAGGING, and I want create a new page with the tag "cat". The vid of the vocabulary is stored in the class attribute 'free_tagging_voc'. What I've been trying to do is the following:

$edit = array();
$edit['title'] = $this->randomName(10);
$edit['body'] = '';
$edit['taxonomy']['tags'][$this->free_tagging_voc] = 'cat';
$this->drupalPostRequest('node/add/page', $edit, 'Submit');

However, this results in the following error message when running the test:

[browser] Setting taxonomy="Array" at [/Applications/MAMP/htdocs/drupal-5.7/sites/all/modules/simpletest/drupal_test_case.php line 111]

Anybody can please tell me what's the right way of doing what I'm trying to do?

Thanks!
Ángel.-

Testing and Quality Assurance

SimpleTest handbook pages - lots of updates to do

catch — Wed, 07 May 2008 11:12:34 +0000

So in the process of writing my first test I found a bunch of documentation discrepancies at http://drupal.org/simpletest

I'm not yet familiar enough with everything to know exactly what's deprecated, what's best practices/optional etc. so starting this to bring them up - then we can just edit stuff directly on the page once it's confirmed it needs changing. Or if they're not salvageable, just archive them quick so we're not presenting misleading information and replace with http://groups.drupal.org/node/11020 when it's ready.

Here's a quick run-down:

http://drupal.org/node/30023 - looks old.

DrupalTestCase should be DrupalWebTestCase ?
drupalModuleEnable($name) - this is still available but should we be using parent setUp($name) for most tests instead? If so this isn't mentioned either in the handbook or the phpdoc.

drupalVariableSet - this is just variable_set() now right? I'm unclear whether this should be used in setUp or at the beginning of the test functions or wherever.

drupalCreateUserRolePerm - should be drupalCreateUser ?

http://drupal.org/node/30010 - also looks old.

DrupalTestCase again.

assertWantedText - should be assertText and assertRaw no?
drupalPostRequest - should be drupalPost

http://drupal.org/node/30008 - more oldness, but also short.

http://drupal.org/node/30021 - ditto

http://drupal.org/node/30004 - archived.

Testing and Quality Assurance

Selenium and Drupal

Souvent22@drupal.org — Sat, 03 May 2008 16:03:04 +0000

First, for those that don't want to read the full post, here's the "30-second elevator speech":
This post to to discuss using Seleinum with Drupal. Specifically using Maven to run the selenium tests and writing the Selenium tests in Java.
The example code be downloaded from the Workhabit Inc. public repository here: https://svn.workhabit.com/svn/public/drupal/selenium/trunk
One must have the following installed to run:
1. Firefox
2. Java 1.5+
3. Maven
All 3 are easy to install and aquire. Once you download the code from the repository, you can just navigate to the directory and run:
mvn -P drupal integration-test
Maven and writing tests are beyond the scope of this blog post.
I'm writing this up for comments, thoughts, and hopefully for others to use and/or contribute.

Ok, now the long winded version:

I've been in the Java world for a bit and I'm now bringing my self back into the PHP/Drupal realm.
With that said on my last project we had to validate ALL code with unit tests, and then we did fully functional integration tests using Selenium.

The Build management tool we used was Maven ( way past the scope of discussion here ). However, what I have done is GREATLY simplified down what we used into a very short, simple, and small file which allows one to run Selenium tests out of the box. It uses Maven/Java to do the tests...but, that's neither here nor there.
It is available here: https://svn.workhabit.com/svn/public/drupal/selenium/trunk

I know that there is a Selenium module for Drupal...however when I thought about this, I also felt that this was an inherit problem. This dawned on me when I read the link that there is a patch to port it to 5.x. There in lies the problem. As we move versions/etc., we're updating the testing frame-work instep with the item we're testing. E.g. PHPUnit may be at 2.3, and we can still use PHPUnit 2.3 for Drupal 4.7, 5.1, 6.x, etc. However, by tightly integrating the testing frame work into the item we are testing, I feel one is creating a cyclic-redundancy in which the end result is chaos.

Goals

My only goals of this project is 2 fold:

Give a solid base so that anyone can download the drupal-seleinum project and write tests for their drupal site.
The community should have an entire suite of tests to test the Drupal installation as a base install ( I have currently started these in the com.workhabit.drupal.selenium.core package. Inside the package, it think they should be split into their subsystems, e.g. Authentication, Taxonomy, CCK, etc.).

Why Java/Maven?

Well, there are a number of reasons; I will not list/discuss all of them here as they are not really part of this post. But on the short list; Java allows one to set break points where Selenium is running to see why a test may be failing ( it should be noted you can also do this with PHP; if you setup your IDE correct and get xdebug installed/setup correctly also).
Also, by using Maven, one doesn't have to worry about dependencies with the unit testing suite (versions, etc.) as Maven takes care of all of that. Java/Maven will also allow us to do more in terms of continuous-build integration. Further, writing tests in Java is not that much harder/different that writing them in PHP. The Selenium IDE gives code in many languages (HTML, PHP, Java, Ruby, etc.) so anyone should be able to create tests easily. I'm hoping to write a post of how to write/add tests in Java. Most of the hard work has already been setup in the code, so all one has to do for a new test pretty much is create a new class and create an annotated java method, such as:

@Test
public void myDrupalTest() {
  BROWSER.open("/path/to/my/site");
  // ....other browser test code from Selenium IDE
  // ...Assertions
  return;
}

Also, in the coming days, I shall be adding to this suite a "MockSoapServer". This will allow you to hit a webservice in point (in this case the MockSoapServer), and it will return "canned" results. Maven will fire up both the Selenium server AND the MockSoapServer BEFORE running the test suites. It will then create a report and take down both after the tests are run.

Environment Setup

First things first, you will need to have Maven installed on your local machine in order to use the Drupal-Seleinum tests. Installing maven is very simple and straight forward. See the Apache Maven Site for more information on installing Maven.

Setup your Tests

So, lets be real; Usually when we are developing, we are changing some of the default workings of Drupal; So we will need to have a some-what custom test suite for each site we develop; or at least we will be extending the tests with more/new tests.

*EXPORT* the base Selenium maven build from the Workhabit repository.
svn export https://svn.workhabit.com/svn/public/sandbox/earnest/drupal/drupal-selenium
Update the POM (If you are unfamiliar with Maven, ignore this step)
You can update the POM if you want to better describe you project.
Modify the "setup.xml"

You will notice when you download the code that there is a file called "setup-example.xml" in the src/test/resources folder. This file is read by the test suite and contains some general settings regarding your site. The settings are pretty self explanatory. The main ones you will need to change out of the box are:
- siteRootPath
- siteDomain
- testUsername
- testPassword
I will hopefully be documenting the settings further, but for now, I apologize that the documentation is a bit sparse.
Test to see that you have everything setup correctly by running a maven integration test (Note: All tests are done with the drupal plugin)
mvn -P drupal integration-test
This should run the DrupalLoginTest which will bring up your Drupal site, and login a user.
Add the package to your own repository.
svn add drupal-selenium && svn commit -m "Adding the drupal-selenium package to my site"
You now have a customized maven/Selenium test suite for the project that you are working on.

Usability

My first SimpleTest

catch — Wed, 30 Apr 2008 15:07:21 +0000

Having not written a single test yet I figured it'd be a good idea to document the process, and I happen to have an issue in the queue which is perfect for doing so.

Background:
* Access rules were removed from core
* Statistics module had a direct call to the {access} table which I knew nothing bout, so is now broken
* Dries said "We should probably write a test for it too so we don't reintroduce this problem."

Everything I know about unit testing has been from the past year or so as it's been discussed in relation to Drupal. I've read odd bits of the docs, read through some core tests, but not written any tests yet or played around with simpletest.module much. I imagine this is the same for a lot of people.

These are any issues I managed to come up with before trying to write the patch, note I'm being deliberately dumb about some of this to try to find what the roadblocks are, and if I appear to be talking to myself, then I probably am.

IP blocking doesn't have any tests existing tests for it within system.module (because that patch (by me) got accepted without tests) - so nothing to build on there.
Statistics.module doesn't have any existing tests either (issue here: http://drupal.org/node/253121)

(These are both temporary conditions hopefully, but important at this stage of trying to get people involved. If I didn't already know what was going on with this, Dries' statement would have stumped me completely.)

There's a whole bunch of steps - like enabling statistics module, then enabling access logging and having some data in two different tables before you can test the page in question. I know you can set variables and create nodes and users cleanly but do I insert arbitrary data into the accesslog and blocked_ips table or just test if the page displays ok or what?

- I couldn't find the answer to this in the docs, so asked chx, who confirmed I need to use db_query() to insert/delete the data manually for these cases (in the absence of mocking). We probably need to come up with some documentation for that - I can think of {accesslog}, {watchdog}, maybe {history} where such information might need to be faked.

The patch to fix the issue changes 10 lines of code, the SimpleTest is going to be a lot more than this.

- if there'd been a SimpleTest for this already I wouldn't be writing a 10 line patch to fix something I broke writing a much bigger one that's already committed. This is why testing is good.

Do I put this test in system.test or statistics.test?

- I think this particular issue needs an integration test and we don't have any of those yet. So I'll put it in statistics, or split it between the two for now.

---
This is a very rough log of the process of actually writing a test for the first time:

Stuff I looked at in order of frequency:
1. core tests
2. simpletest handbook page.

Since there was no statistics.test to work with I copied user.test into a new file to get the basic structure to fill in.

Not entirely clear whether I should be using setUp('module') or drupalModuleEnable. The handbook says drupalModuleEnable but the tests say setUp() - so I went for setUp().

drupalCreateUser() is entirely self explanatory, which is nice. Handbook says I should've used drupalCreateUserRolePerm though - which appears to be deprecated as well.

I have to insert some dummy accesslog information - only way to do this is via db_query(), meh.

Handbook says use $this->drupalVariableSet($name, $value) but I picked up from somewhere that this is deprecated, and the tests say the same thing, so I just use variable_set.

Using the internal browser is the easy bit - do stuff, see what happens, do more stuff, see what happens.

Since there's not existing tests for this bit of core it's hard to know how much or how little to test. I figure I'll test as much as possible and cut it down later if told, for practice if nothing else.

Can't see an obvious way to test whether the #default_value in my form is getting set or not, will skip that for now.

So now I have my first test ready to try out. I wasn't on my development machine when writing this so as yet I've just got something completely untested which vaguely resembles some other tests.

1st run: 19 passes, 14 fails and 0 exceptions.

Not so bad for running blind. Turned out I'd missed a capital letter in an assertText. s/block/Block.

Nice to see what the error is, which line etc., make things much easier to fix.

Fix that, next run:

23 passes, 11 fails and 2 exceptions.

Ok so now it looks like I've got the drupalPost messed up somehow. Ah - no I haven't, I'm rolling this test along with a patch, and I'd done something stupid with my menu definition - so my test is actually failing due to my sloppy code!!

Fix that and check manually that my menu item is appearing this time, one more pass on running the test.

24 passes, 8 fails and 2 exceptions.

However I've got two exceptions - looks like I'm using assertRaw/assertText incorrectly or some other issue - fiddling about with it didn't do me any good, so time to ask for help on IRC.

More fiddling, I was using 12.34.56.789 as my IP address for testing. This, of course, isn't a valid IP address, so it's not getting saved, because the validation works. That was stupid. Note to self - follow the exact steps you're trying to test manually too.

31 passes, 3 fails and 4 exceptions.

Got it to all passes, no fails but still 4 exceptions. Turns out clickLink() isn't handling query strings quite right: http://drupal.org/node/255613

Final test is on this issue: http://drupal.org/node/248436

Testing and Quality Assurance

broken tests?

pwolanin — Wed, 30 Apr 2008 01:09:43 +0000

see: http://drupal.org/node/252920

This profile module test suite also has a lot of redundant code.

Are other tests broken? Is it possible that I'm the first one noticing this?

Testing and Quality Assurance

HOWTO: Submit tests with your patch

webchick — Sat, 26 Apr 2008 04:58:27 +0000

Note: This document is under development and targeted towards Drupal 7

What is Automated Testing?

Automated testing encompasses UI, API and unit tests. It's a technique for testing small sections of code to ensure it works as expected. For example, a UI test might check a form submission process to ensure form display, validation of input and submission works as expected. A unit test might test a function for checking street addresses by passing a variety of well-formed and mal-formed addresses to ensure it handles them properly.

Automated tests are used to aid debugging, ensure compatibility with other elements of the application, and assist in maintaining backward compatibility as code develops.

Why Test?

Modules and patches can be quickly reviewed and committed.
Tests provide insight into how code is expected to perform.
Tests help avoid introducing bugs when developing new features or modifying existing code.
Code that has been developed with testing in mind tends to be easier to maintain and understand.
Bug patches submitted with a test ensure the bug remains fixed.

SimpleTest: A Tool for Unit Testing

Drupal comes with the SimpleTest module, which is based on the SimpleTest framework. The SimpleTest module is off by default and can be enabled through the module administration screen, although it's recommended that the SimpleTest module only be enabled on development systems, as unit tests may affect data in unexpected ways.

Types of Tests

There are three types of tests:

UI tests: These are "black box" tests that simulate the user clicking on various links and buttons in order to test module functionality.
API tests: For API functions that aren't easily represented in the UI, functions that are called by other modules, and so on, API-level tests may also be written. These are "white box" tests which will check database state and so on in order to ensure that APIs are working as expected. They can also serve as a form of documentation for how the API is supposed to behave, as they serve as examples of how functions are supposed to work.
Unit tests: These types of tests test a part of the system in isolation from other parts so that system inter-dependencies don't come into play. In most cases, this requires mocking functions (via runkit or other methods).

The Drupal community is initially focusing on UI and API testing in order to get maximum coverage in those areas, with as low a barrier to writing and running tests as possible.

This tutorial will focus on writing both API and UI level tests.

Where do test files go and what do I call them?

Most of the time, you want to test a single module, in which case you simply create a file called module.test in the module's directory.

One exception to this rule is for various core library files, such as those files found in the /includes directory. For these you would create a file called filename.test in the /includes directory.

Modules that wish to have multiple test files (for example, both a .test file for the module itself and a .test file to ensure the module conforms to external RFCs or specifications) may also create a "tests" subdirectory and place multiple .test files within it.

Taking a tour: What's in a test file?

TestCase classes (test groupings)

Each .test file must contain one or more TestCase classes which hold groups of tests. TestCase classes appear in the testing module user interface, checkboxes for each class allow different combinations to be run for each test run.

An example TestCase class declaration is:

<?php
class ModuleNameTestCase extends DrupalWebTestCase {
  ...
}
?>

A typical simple module can get away with having only one TestCase class to hold all of its tests. However, there are circumstances where you may want to add additional TestCase classes:

If it's conceivable that someone would want to test different parts of your module independently of one another; for example, it makes sense to have different test groupings for system module's module-related tests vs. its theme-related tests.
If there would be performance impacts of running all of the tests at once (for example, all node module or all database system tests), it makes sense to split them up into separate groupings.
If the module just plain does a lot of stuff and you want to break things up a bit to make them more manageable, additional test groupings also can make sense.

When creating multiple testing groups in a single .test file, use a ModuleNameDescriptionOfTestTestCase naming convention instead (for example, DatabaseSelectTestCase, DatabaseFetchTestCase, DatabaseInsertTestCase, etc.).

getInfo() method (metadata about test group)

Each TestCase class requires a getInfo() method, which returns meta-data about the test: its name, description, and what "group" it belongs to (such as the module it's written for).

An example getInfo() method is:

<?php
  // Inside TestCase class...

/** * Implementation of getInfo(). */ function getInfo() { return array( // 'name' should start with what is being tested (menu item) followed by what about it // is being tested (creation/deletion). 'name' => t('Noun verbion'), // 'description' should be one or more complete sentences that provide more details on what // exactly is being tested. 'description' => t('Create a custom whoozit, add thingies to the stuff, and clean up afterwards.'), // 'group' should be a logical grouping of test cases, like a category. In most cases, that // is the module the test case is for. 'group' => t('Module'), ); } ?>

Test methods (the tests themselves)

The vast bulk of .test files are the test methods, which contain the tests themselves. At the heart of all test methods are assertions.

A list of all available assertions can be found at http://drupal.org/node/265828

Here are example test methods in a typical TestCase:

<?php
  // Inside TestCase class...
  /**
   * One-sentence description of test.
   */
  function testModuleNameTaskVariation() {
    // Code that does something to be tested.
    // Test that the code did what it was supposed to do.
    $this->assertEquals($some_value, $another_value, t('Testing equality of some value and another value'));
  }

/** * One-sentence description of test. */ function testModuleNameTaskVariation() { // ... $this->assertNotEquals($some_value, $another_value, t('Testing inequality of some value and another value')); } ?>

Some guidelines about test methods:

Aim for each test method to test only one thing. This makes it much easier to hone in on what was being tested when failures appear. For example, rather than an enormous testUserLogin() function with several variations, instead do testUserLoginDuplicateUser() and testUserLoginRequiredFieldsMissing().
For UI tests, name the methods testModuleVerbVariation(), such as testUserLoginDuplicateUser().
For API tests, name the methods testFunctionNameVariation, such as testVariableGetMissingValue().

setUp()/tearDown() methods (setup and/or cleanup after tests)

setUp() and tearDown() are two optional methods that perform setup tasks and cleanup tasks, respectively. These methods run before and after each test method.

An example:

<?php
  // Inside TestCase class...
  /**
   * Implementation of setUp().
   */
  function setUp() {
    // The first thing a setUp() method should always do is call its parent setUp() method.
    // If you need to enable any modules (including the one being tested),
    // add them as function parameters.
    parent::setUp('statistics');
    // Next, perform any required steps for the test methods within this test grouping.
    variable_set('some_variable');
  }
  /**
   * Implementation of tearDown().
   */
  function tearDown() {
    // Perform any clean-up tasks.
    variable_del('some_variable');

// The last thing a tearDown() method should always do is call its parent tearDown() method. parent::tearDown(); } ?>

If you only call parent: setUp(); and nothing else in your setUp() function, then you do not need to call tearDown() since SimpleTest module handles this basic cleanup automatically.

References on unit testing and Drupal:

Testing and Quality Assurance

Tests in core: What's next?

flobruit — Thu, 24 Apr 2008 07:46:28 +0000

We now have a testing framework in Drupal core. What we need to move forward is some structure that makes writing tests easy and efficient so that we can go towards our 100% code coverage goal without wasting development resources.

After today's IRC conversation on the topic of unit testing, the decision was taken to implement unit tests the same way as functional tests are currently implemented, without mock objects or functional overrides at runtime (see this issue for an explanation: http://groups.drupal.org/node/10957). However, the discussion also touched on many key points that are relevant no matter what approach we take, so here's a summary for those who weren't there.

One of the main outcomes was the need for a better organization of the various kinds of tests that we have. With a clear structure in place, we could put test files in place with minimal code, thus making it easier for contributors to create patches without having to figure out where to add files or how to trick cvs into creating a patch for new files. These basic files would also make it easier to point people to simple tasks that can be processed in a short amount of time without the need for an extensive understanding of the code.

In order for tests to be easy to write, they must be easy to read and easy to understand. The existing tests are the best documentation that we have for contributors who are new to automated testing. We currently have many tests that try to cover a whole module's functionalities with a long list of assertions. The problem with such tests is that they make it hard to track down what is happening when a test fails (even if we know what assertion failed, we might need to debug the code that sets the fixture – the environment in which the test is run). It is also a lot easier to create a small test than it is to make a small addition to an existing mastodon. In other words, tests need to be refactored, just like code.

Besides reorganizing the structure of the tests, it is also important to make a distinction between the different kinds of tests. It's not because we are using the same mechanism to process “unit” tests as we do for functional tests that they are the same thing. We currently have tests that simulate specific sequences of user actions (like user.test), tests that simulate the execution of code with specific parameters (like search.test), and some that are somewhere in between (path.test). In order to better structure our tests, we need to be able to indicate what kind of simulation they perform. Should unit tests be in separate files, separate classes, or even simply separate methods? Wherever we put that separation, we also need to make sure that the naming conventions reflect it.

Finally, making Drupal testable requires refactoring code, and refactoring code breaks existing tests, but this is what is supposed to happen. The whole reason to use automated tests is to be agile and make this kind of overhaul possible by making us aware of how we break things. Converting Drupal to a fully tested framework is a big task, but the work we do now will make future work easier and our testing system (both the testing framework and the tests themselves) will get more stable, to the point where we can make big improvements without breaking everything.

Note: Credits for many of these points go to the people involved in the IRC discussion (mainly webchick and chx), but I've mixed in my own opinion so criticisms go to me.

Testing and Quality Assurance

Unit testing plan

chx — Wed, 23 Apr 2008 15:21:34 +0000

We are pretty ahead with functional tests but unit tests are a completely different problem. The function we want to test calls other functions and so on. While theoretically one can unit test the leaves of the call tree and work from there, saying "drupal_validate_utf8 is already tested and we know it works. So when testing check_plain, I can rely on the fact that drupal_validate_utf8 works as it should so I will just call check_plain with various values and test with assertEqual whether the return value is what I want." However, you need to unit test every code flow and some parts might be incredibly hard to reach. For example, drupal_get_form calls form_get_cache, drupal_prepare_form, drupal_process_form, form_set_cache, drupal_rebuild_form, drupal_render_form. Even if all these called functions (and everything "below") them are unit tested, providing an environment (database, other loaded functions for hooks and various callbacks, files) that will actually result in certain variables having the value we want might be incredibly difficult.

The traditional solution is to create mock objects, in our case, mock functions. Create a version of drupal_prepare_form which returns exactly the value we want for a particular test and move on. This is not easy with PHP. Once the function is loaded, you can't override it unless you are using runkit which is a PHP extension from PECL. I am aware of two solutions besides runkit, both of which requires Drupal being parsed.

Put every function in a separate file, when you want to use the mock version of a function, load that instead of the original. This is just a bit more difficult than runkit, because you can override only on loadtime not runtime but at least you can override.
Generate mock functions for every Drupal function, load these into memory. Generate a copy of Drupal functions with an original_ prefix, load these too. Call the original function which will in turn will call the mock versions.

Advantages/disadvantages:

When you need the original again to be tested, you need a different PHP instance. However, this also allows to change the mock functions from one test to the next. You can even generate them on the fly, either by copying files or just running eval. So you can have a drupal_validate_utf8 which is mocked to a function which returns TRUE and you can have a version which returns FALSE.
As you need the mock versions to do different (but fixed!) things from one test to the next and unlike the previous ones, these stay in memory, you probably need to call a central function -- which I call the clearinghouse -- which says what this function needs to do.

You basically need to encode some way "I need db_fetch_object to return an object on the first two calls and FALSE on the third". You can call a "createMockFunction" method with the necessary arguments which describe such a thing. Or you can

<?php
array(
  'incoming' => 'db_fetch_object',
  'return' => $object1,
),
array(
  'incoming' => 'db_fetch_object',
  'return' => $object2,
),
array(
  'incoming' => 'db_fetch_object',
  'return' => FALSE,
)
?>

The first solution described above better yields to the "createMockFunction" function which in this case would probably load the necessary code from some include, slap the necessary function header on it and eval. The second solution works better with the array as it literally describes what the central function can expect as the test runs. However, the array can be iterated and createMockFunction calls can be issued for the first solution. Or you can build the array as createMockFunction calls arrive and then as the test starts, the central function can pull the array in and act accordingly. createMockFunction is a bit more like what other systems do. The array version is probably a bit more easier to write on the other hand. The difference is not that big.

And of course, you can always say no to mock functions and just test with the assumption that you have the underlying functions tested and set up the surrounding environment (database, loaded functions, files) so that the code flows the way you wanted. This is the worst, IMO. The load time overrides (first method) is quite a bit cumbersome despite it is what other systems do. Those systems, however, are likely using classes which are more easily overrideable. The central clearinghouse seems to be best suited for Drupal even if the tests are strange arrays.

Note that this is not unlike what happened to our unique form API: that also uses arrays, it's totally unique to Drupal but at the end it proved itself to be a very strong tool.

Testing and Quality Assurance

Unit Test Failure Plan

dlhubler — Mon, 14 Apr 2008 03:41:40 +0000

I ran all the unit tests and I get
Test cases run: 57/58, Passes: 3123, Failures: 282, Exceptions: 34
Is this what other folks get?

Question 1: Do we need to coordinate the fixing of them now?
Question 2: Are we striving for 0 failures at the end of all this?

For me question 2 comes first. Unless we decide we're striving for 0 failures (and I think we should) then if someone puts the time in now to fix tests, tomorrow they may break again and no one will notice. If no one notices, we cannot correlate commits to unit test failures which is an important part of testing, if not the most important part.

Testing and Quality Assurance

Drupal 7 Automated Testing Sprint in Paris Funding proposal

Amazon — Wed, 02 Apr 2008 17:15:57 +0000

Summary: 20 000 patches submitted by 1200 contributors for Drupal 7 will require:
* testing tools development with patch testing automation
* 1200 core contributors to be trained on how to write and maintain core tests

The tantalizing goal The Drupal project lead has presented an interesting challenge to the Drupal project's core development contributors. If contributors add 100% testing coverage to Drupal 7, they can reap the benefits of automated testing by having four more months of feature improvements in the development cycle. Dries has left the Drupal community drooling for more development time.

Current state

The problem with Drupal testing

The Drupal community has already begun to embrace testing for Drupal 7. Core contributors are beginning to provide additional tests, or improvements to existing tests along with their patches. If the Drupal 6 development cycle grew to 9425 patches submitted by 741 unique developers, then it's possible the Drupal 7 development cycle will have 20 000 patches with 1200 or more core contributors based on growth rates between previous releases. This presents two challenges for the Drupal community. First, test creation tools, patch testing automation, functional test creation, unit test creation, and maintenance of those tests against the HEAD branch present serious systems integration and collaboration challenges. Second, assuming this complex infrastructure is working flawlessly then we must go about training 1200 potential core contributors. Contributors will need to make a huge initial investment in test driven development and change the way they contribute to Drupal. This may takes years. We are going to need a full suite of development best practices and techniques to succeed.

The primarily method for developer communication is the project module that runs on Drupal.org. We had approximately 200 000 issues and follow-ups in 2007. Thanks to great maintenance by Derek Wright, Chad Phillips, Adam Light, and Jimmy Berry the project module should scale to handle significant growth. Our infrastructure team lead by Gerhard Killesreiter with lots of leadership by Narayan Newton and backed by reviews from Jeremy Andrews and David Strauss should allow the Drupal infrastructure to scale to meet increases in both patch submissions and testing.

Stephen Peck and the documentation team continue to make great progress on the huge challenge of gardening Drupal's documentation. They will continue to provide a solid framework for patching and test creation documentation to be created. The popular Drupal dojo is a great model for trainin. Charlie Gordon and as well as others will step up to bootstrap the Drupal communities test creation and test maintenance instruction with Drupal dojo. Don't get me wrong, none of these efforts are without their own great challenges, and we are relying on dozens of contributors to perform as admirably as they have in the past. The beating heart of Drupal is the 24/7 banter and friendly collaborative discussion that go into IRC channel #drupal, now one of half a dozen specialized Drupal channels. The #drupal IRC channel will continue to be the friendly introduction for person to person support, and test development training, the community does so well.

However, there one is technique the Drupal community has relied on to make great progress on in the past that we can use again. That's the Drupal developer sprint. We've done sprints at:

Drupalcon Amsterdam 1 day bug-fix sprint for 4.7
OSCMS Yahoo had a sprint
St. Charles, Illinois lead to Forms API III
At Drupalcon Barcelona, the CCK redesign efforts are proving to be successful
The Data architecture sprint in Chicago led to Database TNG and the registry patches

A testing sprint could be critical technique for ensuring the complex infrastructure we are building is ready to support 100% test coverage.

Solution

We are currently proposing that we conduct a testing sprint with at least four key testing infrastructure developers at DrupalCamp Paris. The sprint offers a mid way point between testing contributors Karoly and Rok, who are both located in Eastern Europe and who recently traveled to Drupalcon Boston.

The sprint aims to accelerate progress across 5 different testing projects that will support the Drupal 7 testing initiative. In person communication, will allow for decisions and problems solving to occur at an accelerated rate.

Outcomes

integrated plan for testing, including
- unit testing
- functional testing
- automated patch testing
- javascript testing plan
- improve automated test creation tools
- infrastructure testing plan
- coverage metrics
find ways to improve consistency and quality of tests
better understanding of current state of testing
increased test coverage for critical areas; larger number and variety of tests
infrastructure for accepting and integrating community developed tests
improvements for http://testing.drupal.org

Deliverables

high-level plan for testing
standards / guidelines / best practices / templates
significant number of new tests to increase coverage, focusing on key areas
review / state of current tests report
plan for and initial implementation for test submission and management process

Budget

There are many expenses with international travel and meeting. We've found one opportunity for testing sprinters to meet. DrupalCamp Paris is taking place April 19th. The organizers of DrupalCamp Paris, Ori and Louis from AF83, have been working with us to provide space, food, and partial accommodations.

To bring 4 sprinters together for Drupalcamp Paris we estimate airfare for 4 people to be $3000 US. For 6 people, $5000.
Hotel accommodation should be $400 for Friday, Saturday, and Sunday.
Some meals will be provided by AF83, and the Drupalcamp Paris organizers
Developers are volunteering their time. The developers are taking 1-2 days off work to travel to the event and volunteering for a full weekend.

Logistics

Map of sprint
Arrive the evening of the 18th in Paris. Depart Monday afternoon the 21st.
List of potential hotels.
Friday arrival times by time of arrival from the wiki page:

Morning:
Charlie: Arrives Thursday morning, ~7-8 AM. Will likely be zonked at that time, though, as it will feel ~midnight. Staying with grandparents.
Douglas: Arrive Fri 825 AM. staying at Hotel Cyrnos, leave Monday 1:40PM
Kevin: Arrive Friday 11:05 AM CDG. staying at Hotel Cyrnos, leave Monday 12:55 PM CDG

Afternoon:
Jimmy: Arrive Friday 2:55 pm (local) staying at Hotel Acadia Opera, and will leave 12:50 pm (local) on Monday.

Evening, saturday
Rok: Arrive Friday 9pm CDG staying at Hotel Acadia Opera, leave Monday 9.20pm CDG
Karoly: Arrive Friday 9pm CDG staying at Hotel Acadia Opera, leave Monday 8:20pm CDG
Dries: 12pm saturday

Sprinters

Karoli Negyesi - confirmed, funded by NowPublic, country Hungary
Rok Žlender - confirmed, funded by NowPublic, country Slovenia
Charlie Gordon - accomodation secured, Google sponsored, country USA
Jimmy Berry - Confirmed, sponsored by Google, USA
Kevin Bridges - Confirmed, funding by pingVision, USA
Douglas Hubler - confirmed, funding by Acquia, USA
Dries Buytaert - confirmed, funded by Acquia, Belgium

Results

Testing and Quality Assurance

Unit -vs- UI testing

Harry Slaughter — Tue, 25 Mar 2008 03:16:29 +0000

Please correct me if I'm wrong, but it seems that this group's purpose is to concentrate on unit testing and code coverage. Excellent. This is a definite sign of maturity for Drupal.

But I also see some posts in here that refer more to UI centric testing. Is there yet an effort for organizing this testing as well?

Unit testing is a great way to find problems that may not always be obvious in the UI, but they are not always the best way to catch problems which may be obvious in the UI. For example, the unit tests for the forms API may confirm that the form tokens are being generated properly but not pick up on the fact that another module alters the token (making this up here :) and so the form submission ultimately fails. At the end of the day, you really need to run through use cases to make sure it all works.

Typically, folks will run functional/UI tests by hand (create, edit, delete a node by hand, etc...). This is OK, but not ideal. Manual testing is time consuming and difficult to reproduce.

I've done testing of applications where automation was not an option, and as standardized as I tried to make the list of tests I ran, they were never run the same way twice, leaving little crevices for regression to creep through.

I believe we should consider trying to automate UI testing as well as unit testing. This would give us a gorgeous set of regression tests.

I guess my main question is whether this is the place/time to bring that up. I don't think it would take any attention/resources away from the unit testing efforts here.

Testing and Quality Assurance

SimpleTest Roadmap for Drupal 7

boombatower@drupal.org — Mon, 24 Mar 2008 20:45:29 +0000

There are many things that need to be done to SimpleTest is to provide full test coverage, as requested by Dries, and be placed in the core. A lot of work has already been completed and we are making great strives towards making this a reality, but there is still much to do.

I have provided a general overview of what needs to be accomplished.

Complete functional tests. (list)
Review functional tests and ensure that they pass against HEAD and test necessary functionality.
Plan unit testing
Generate stub unit tests using SimpleTest Unit Testing
Write unit tests.

There are a number of tasks that need to be completed to accomplish the goals above. This list is by no means final and will most likely grow with time.

~~#244741: Remove drupalVariableSet~~
~~#244701: Internal browser doesn't select first option and ignores selected options~~
~~#243769: Reset Environment~~
~~#243367: System tests are broken~~
#243532: Parse drupal errors
~~#242958: Clean up the modules code a bit~~
~~#242843: Clean up database table from crashed tests~~
#242934: Polish profile test
~~#239358: Get tests working for upload module~~
~~#241956: Do not call exit in simpletest module for CLI~~
#242069: Simpletest should include exception data in reporting
~~#242474: Using fnmatch breaks simpletest on windows~~
~~#240619: Default assertText Message~~
~~#240590: Remove various cleanup variables from DrupalTestCa~~
~~#240590: Remove depreciated method names~~
~~#238379: A little cleanup of the new UA code~~
~~#239565: Multiple test functions and db_prefixing cause error~~
~~#238640: Assert function for checking the non-existance of fields~~
~~#238538: Story is now article~~
~~#203784: Upload settings test~~
#234409: Create Group Convention
#212351: Polish the crap out of SimpleTest module
#237964: Clean up documentaion
#237963: Get coding standards into Coder Module
#234405: Coding Standards
~~#237931: Support for file input type in the new internal browser~~
#233161: Tests for issues
~~#179534: Use a separate test database, transactional/rollback~~
~~#231348: Find a consistent naming convention for the *.test~~
~~#235013: desc is not Drupalish, use description~~
~~#237940: drupalPost throws exception in new browser~~
~~#237934: assertFieldById throws exception~~
~~#237667: get_info to getInfo~~
~~#237407: assertField() is broken~~
~~#237369: Clean up user-related functions~~
~~#236535: Consistent usage of t()~~
~~#236131: Create dummy files that can be used by tests for u~~
~~#232109: Replace SimpleTest browser with our own~~
~~#59505: Taxonomy tests need updating~~
~~#209297: New param in drupalGet~~
~~#230138: Absence defaults['created'] in drupalCreateNode~~
~~#229179: Wrong format in drupalCreateNode~~
~~Large number of test issues.~~

Below is a list of issues that need to be decided.

#214028: Break Tests up by method
~~#199880: Call to undefined method BadGroupTest::getTestIns~~
#214570: Replicate 'Begin' button at the top of the page
#212304: drupalCreateNode does not use the internal browser
#230269: Generate urls for running tests directly
#212415: Use hook_requirements() rather than odd simpletest
#233161: Tests for issues

The following are fixed issues that have comments that need to be looked at for possible problems found through testing.

#245955: Review: BlogAPI Module Test
#244763: Review: User Module Test
#245904: XMLRPC doesn't check for NULL error object
#242475: Help module test
#241467: DBLog module test
#240644: Blog module test
#244602: Forum test
#237336: Url Aliases no longer work

Things to remember to fix/change.

#244763: Move unit tests out of user functional tests

Testing and Quality Assurance

Develop an xss and sql injection scanner based on SimpleTest

ingo86 — Fri, 14 Mar 2008 15:02:03 +0000

What I wanna develop for SOC 2008 is a module called security (or add security function to simpletest existing module) to enable users checking their drupal installation against xss and sql injection vulnerabilities.
It will be also good for module developers, in fact they can check their module before submitting them to drupal website. Users could be more protected against vulnerabilities that became from third part modules.
The objective of this work is to realize automated penetration test on drupal installation.

It will be based upon SimpleTest, already used by Rasmus (php core developer) to develop his own closed source xss scanner. SimpleTest is a jUnit similar library written for php.

My module could easily been extended to add more functionalities about security, but basically I think that this two are the most important.
If someone has functionality ideas to improve my project and make it better I'm here, listening for more proposal.

Coding Standards and Performance Optimization

Drupal QA in the Next Year

Robin Monks@drupal.org — Wed, 12 Mar 2008 21:16:45 +0000

By: Robin Monks

Introduction

Drupal is community. It’s a community of visionaries that make Drupal what it is, we all want to see Drupal be more powerful, more flexible, and just plain better than the competition!

That said we need to be aware that the more popular our product gets, the more we need to test for various scenarios; and not just from the developers point of view, but from the users. This is where Quality Assurance comes into play. QA can help give Drupal core that extra level of stability and sanity, and the good news is we’ve already taken the first steps.

Non-Interactive Functionality Testing (Unit Tests) [Already Happening]

· http://drupal.org/project/simpletest

· http://drupal.org/project/Drupaltestbed

· http://drupal.org/project/project_issue_file_test (aka. PIFT)

Non-interactive testing via projects such as http://testing.drupal.org will go a long ways toward eliminating the need for developers to manually test each submitted patch for syntax quality and general “does it run?”-ability. It will be sure to allow patches to be processed much faster.

Smoke Testing [Concept]

We need a list of common tasks that volunteers can perform on various host and browser environments to test for broken or unexpected functionality. I plan to make some further guidelines, and perhaps some tools, for this shortly.

User Interface Quality [Partially Complete]

The g.d.o Usability Group has made great progress to this goal, here is some of their fine work:

· Views 2.0 UI guide

· Handbook style guide

· Documentation writer’s guide

Here are some things I’d like to see added:

· Style guide for help text and prompts in Drupal (eg, the confusing password strength “error” during install)

· Module and form description style guide

· A general UI element style guide a la Gnome

Disability Usability

We need to ensure all Drupal features can be access by those with screen readers, or with color blindness.

Full Standard Compliance [Partially Complete]

We need to monitor to ensure that current Drupal features (HTML, JS, CSS) meet existing standards; we also need to make sure as we enter the world of the semantic web, where data is free-for-all, that we also stick to know standards and conventions so as to allow the presented data to be as interoperable as possible.

Community-powered Quality Events [Concept]

Bug-a-tons, bugdays; prizes for those who process the most bugs, etc. These could all be means to get more people involved in the issue queues, and to enhance the overall quality of Drupal.

Central Source of Information [Concept]

There needs to be a central location where everyone interested in Drupal quality can find out information, and participate. The great thing about quality testing is those involved don’t necessarily have to be developers. It’s a great way to get more people involved in the Drupal development process!

I think the g.d.o “Quality Assurance” group has the potential to be that place. I’ll be making myself active in that group and will help co-ordinate changes to implement the items in this “rough guide” I’ve outlined. None of it should be considered “official”, but I believe it will serve as a great guide for future QA efforts.

Quality Assurance

Test stub generator...

lego1974 — Wed, 12 Mar 2008 21:05:58 +0000

Hello, I've written a perl script (fastens fire-proof suit) that generates module files as well as a test file complete with a test class and stubbed methods. I've posted the code as an issue to the simpletest module as webchick suggested: http://drupal.org/node/233261

From the documentation:

Features:
- creates a module folder and associated file skeletons
- generates a test class and test methods and places them in a test file
recognized by SimpleTest. This currently only covers unit tests,
and not higher level tests like functional or integration tests.
- can be re-run throughout module development
- if you add a new function to a module, rerun the script and a test
stub will be generated and placed in the appropriate test class
- works with existing modules
- adds hook_simpletest to an existing module

Issues:
- test stubs intentionally fail. I tried replicating PHPUnit's
TestNotImplementedException, but I didn't find a similar facility

Assumptions:
- the drupal coding standard is followed (generally, 1 tab == 2 spaces)
- a single module is defined per directory
- the module has the same name as the directory

Features Not Included:
- getters and setters for classes defined in module file
- class handling in general is not working, but the basic code is in place

TODO:
- complete handling for classes
- add handling for function removal by developer
- make template variables files - is this necessary?
- create CRUD scaffolds a la Ruby on Rails
- improve test coverage to include suites and further automation

Anyway, I hope you find this useful. I'm using it internally at my company. It eliminates a lot of the tedium of creating the tests. Note that it simply covers unit tests and not functional/regression/integration tests. There's lots that could be added. I'm hoping to get lots of feedback on it.

-Andy

Testing and Quality Assurance

Testing and Quality Assurance

Functional SimpleTest Status - List of all tests

Purpose

How you can help

Maintainer

Status

List

Test State

Rename the "Unit Testing" group to "Testing"

SimpleTest module's faults at first sight

Tests that fail by design

Known test failures

XML output for simpletest requests

Getting Involved in SimpleTest

7.x Related Issues

5.x and 6.x

Undefined index exceptions

Correct simpletest syntax for adding tags to a story/page form?

SimpleTest handbook pages - lots of updates to do

Selenium and Drupal

Goals

Why Java/Maven?

Environment Setup

Setup your Tests

Modify the "setup.xml"

My first SimpleTest

broken tests?

HOWTO: Submit tests with your patch

What is Automated Testing?

Why Test?

SimpleTest: A Tool for Unit Testing

Types of Tests

Where do test files go and what do I call them?

Taking a tour: What's in a test file?

TestCase classes (test groupings)

getInfo() method (metadata about test group)

Test methods (the tests themselves)

setUp()/tearDown() methods (setup and/or cleanup after tests)

References on unit testing and Drupal:

Tests in core: What's next?

Unit testing plan

Unit Test Failure Plan

Drupal 7 Automated Testing Sprint in Paris Funding proposal

Current state

Solution

Outcomes

Deliverables

Budget

Logistics

Sprinters

Results

Unit -vs- UI testing

SimpleTest Roadmap for Drupal 7

Develop an xss and sql injection scanner based on SimpleTest

Drupal QA in the Next Year

Introduction

Non-Interactive Functionality Testing (Unit Tests) [Already Happening]

Smoke Testing [Concept]

User Interface Quality [Partially Complete]

Disability Usability

Full Standard Compliance [Partially Complete]

Community-powered Quality Events [Concept]

Central Source of Information [Concept]

Test stub generator...