Testing and Quality Assurance

Events happening in the community are now at Drupal community events on www.drupal.org.
This group should probably have more organizers. See documentation on this recommendation.

Welcome to the group for the plumbers who keep the community plumbing from leaking. ;)

Testing frameworks such as SimpleTest and Selenium, automated testing, and general community QA initiatives are all on-topic here.

Manual testing is also totally on-topic.

julma's picture

What is biggest drupal project you have ever managed ? (size in man-days)

Less than 50 man-days
30% (27 votes)
Between 50 man-days and 200 man-days
24% (22 votes)
Between 200 and 1000 man-days
29% (26 votes)
More than 1000 man-days
18% (16 votes)
Total votes: 91
dlhubler's picture

Latest test results - updated dailly

http://acquia.com/latest-drupal-test-results

The results are updated around 3 AM Eastern Daylight Timezone every night.

Read more
damien tournoud's picture

SimpleTests that are known to fail on PostgreSQL

Here are the results of running our whole test suite on PostgreSQL 8.3.

  • Actions configuration 30 passes, 8 fails, 0 exceptions
    See http://drupal.org/node/261859
  • Block functionality 57 passes, 1 fail, 0 exceptions
    No known issue (yet!)
  • Blog API functionality 45 passes, 2 fails, 1 exception
    No known issue (yet!)
  • Search engine ranking 21 passes, 5 fails, 20 exceptions
    See http://drupal.org/node/296624
  • Module list functionality 53 passes, 4 fails, 0 exceptions
    No known issue (yet!)
Read more

Updating 6.x-1.x style tests to 6.x-2.x and/or 7.x

In porting the Pathauto tests from the Simpletest 6.x-1.x to 6.x-2.x style I found a few things I thought I would document here in the hopes that they benefit others:

  1. Put the .test file into the root directory of your module. If you had it in modules/tests/module.test it now belongs in modules/module.test
  2. DrupalTestCase is now DrupalWebTestCase
  3. get_info() is now getInfo()
  4. in getInfo, desc is now description
  5. $this->drupalCreateUserRolePerm is now $this->drupalCreateUser(
  6. drupalLoginUser is now drupalLogin
Read more
catch's picture

testing.drupal.org

Splitting the discussion from here:

http://lists.drupal.org/pipermail/development/2008-August/030879.html

afaik, the steps to get this running are:

  1. Abstract comment creation functionality in project_issue.module - http://drupal.org/node/271216
  2. Allow testing.drupal.org to post back to issues and mark as code needs work when patches don't apply.
  3. Get all known failures cleared
  4. Allow testing.drupal.org to post back to issues and mark as code needs work when core tests fail
  5. Extend this to Drupal 6
Read more
niklp's picture

Google Ratproxy - a web application security audit tool

Submitted for perusal by the group, as passed to me by a fellow developer.

"[Google ratproxy is a] semi-automated, largely passive web application security audit tool, optimized for an accurate and sensitive detection, and automatic annotation, of potential problems and security-relevant design patterns based on the observation of existing, user-initiated traffic in complex web 2.0 environments.

Read more

Prep-work for Awesome Testing Party

Great news! The Awesome Testing Party session was accepted at Drupalcon Szeged! Woohoo!! It'll be at 9am on August 28th, but people should probably be there a bit earlier, like 8:30am.

There's lots to prepare, and very little time to do it in! We had a discussion today in #drupal to talk over logistics. Here's where we stand. Please strike things as you get them done. Thanks!

Stuff for lead-up to the event

  • Write a testing hand-out that summarizes stuff from the intro to testing session (flobruit)
  • Do a PR blitz to Drupal Planet about the party, talk about its benefits and goals, how it's going to work, and ask Szeged attendees to bring a box of chocolate with them (webchick) - http://webchick.net/awesome-testing-party
  • Re-run test coverage scripts to see where we stand (catch) - http://coverage.ca.tchpole.net/coverage/html/1
  • Do a screencast about how to write tests (sdboyer)
  • Create a .test template that people can copy/paste from (webchick)
  • Create issues for each candidate test and title them TestingParty08: Description of test" (boombatower and catch) - list here
  • Write up a hand-out describing how testing party will work for participants (webchick)

Stuff for the day of the event

  • Look into getting Hungarian pancakes for breakfast ;) (chx)
  • Make lots of photocopies of the testing handout to bring with (webchick)
  • Go buy some index cards and write all the TestingParty08 issue Node IDs on them (webchick) - they're even multicoloured!
  • Bring chocolate for the big "chocolate stash" at the front of the room (everyone!)

Agenda/workflow

  • Pancake bar as people come in
  • 5-10 demonstration on how to do testing
  • Pair up testing veterans with testing newbies
  • Do test cases for the rest of the time:
    • Get pancakes
    • Get a testing issue
    • Solve and eat testing issue and pancakes, respectively.
    • Submit patch for review.
    • Claim a new task/pancake if you have no tasks at cnw, otherwise tidy up those issues.
    • Repeat.
    • Chocolate fits in here somewhere too.

"Volunteers"

  • cwgordon7
  • webchick
  • sdboyer
  • chx
  • boombatower
  • catch
  • flobruit
  • damz

If you see anything missing from this list, or would like to volunteer to help with any of these items, feel free to edit away. :)

Post-event

To make it easier to Dries to commit the tests and to ensure the quality of the tests the patches should be reviewed by those involved in testing development.

  1. Aggregate testing party patches into bulk patches related to each file/module they test instead of small tasks. (Without "TestingParty08" prefix)
  2. Review the tests while aggregating and ensure that they all at least pass.
  3. Create issues for the bulk patches or fill in existing ones.
Read more

Spring test cleaning - A battleplan

While Dries is away, and at least before the DrupalCon (and its awesome Testing Party!), we really should clean-up the Simpletest module and core tests.

Read more
sdboyer's picture

Test granularity

I'm working on a project that does some fairly crazy extensions to drupal core functionality (biggest single item right now: $user goes from a stdClass object into an actual classed object, and I use OO fun to make a seven-way distinction between different user 'types'), and have resolved to basically take a TDD approach to driving the site forward, given the complexity and breadth of these changes.

Read more
damien tournoud's picture

Rename the "Unit Testing" group to "Testing"

Today the Drupal Testing framework is many an UI ("functional") testing. Should we rename this group to simply "Testing"?

Damien Tournoud
http://drupalfr.org

Read more
internal's picture

SimpleTest module's faults at first sight

1. Too heavy to write simple test

if I changed code in blog_link to remove author's blog link, how can I write a small code to ensure it? logic as

testnode = new node;
node->type = blog;
testlink = blog_link('node', testnode);
assertNotContains('tom's blog', testlink);

2. Not IDE friendly, even for simpletest series.

Using simpletest eclipse plugin, you can't run test file because of the .test extention, have to rename to .php. After renaming, it still can be run because of complex file including.

Any thoughts?

Read more
catch's picture

Tests that fail by design

We're down to under 60 core test failures, and there's at least four RTBC patches in the queue which will reduce that further, meaning we'll be down to 5-6 unresolved issues dealing with core test patch failures pretty soon.

At the moment, there's only one test that fails by design - for the url filter. However that core bug is likely to get fixed before all tests are working again, so it's unlikely to impact on much.

Read more
catch's picture

Known test failures

If you're looking for current status, look at the most recent comment

Since we've still got core test failures, and no automated patch testing, I thought it'd be worth doing a snapshot of tests that pass week by week (or more frequently if there's lots of commits). This allows us to keep a historical overview of where bugs have been introduced into either core code, simpletest.module or tests themselves. It should also hopefully help to reduce duplicate issues about broken tests (since a test failure can be down to any one of those conditions).

Read more
dropcube's picture

XML output for simpletest requests

I have created a feature request at http://drupal.org/node/266220

Feedback please.

Read more
boombatower's picture

Getting Involved in SimpleTest

Now that SimpleTest is apart of Drupal core issues related to the 7.x development have been moved to the core issue queue. If you would like to be involved I have included two links which will filter the issues to only display SimpleTest related issue. You can obviously used the advanced search feature to narrow your results to the SimpleTest related components instead of using the links.

Read more
angeliti's picture

Undefined index exceptions

Hello everyone!

I keep getting 'Undefined index' exceptions in my tests:

Unexpected PHP error [Undefined index: weight] severity [E_NOTICE] in [/Applications/MAMP/htdocs/drupal-5.7/modules/taxonomy/taxonomy.module line 502]

As you all know, they occur in this kind of situations:

$example = array('name' => 'example');
if ($example['weight']) {
  ...
}

One option to avoid this is to check if the variable is set:

if (isset($example['weight'])) {
  ...
}
Read more
angeliti's picture

Correct simpletest syntax for adding tags to a story/page form?

Hello everyone!

I'm having problems when trying to add tags to the story/page creation form using simpletest.

I have a vocabulary which uses FREE TAGGING, and I want create a new page with the tag "cat". The vid of the vocabulary is stored in the class attribute 'free_tagging_voc'. What I've been trying to do is the following:

$edit = array();
$edit['title'] = $this->randomName(10);
$edit['body'] = '';
$edit['taxonomy']['tags'][$this->free_tagging_voc] = 'cat';
$this->drupalPostRequest('node/add/page', $edit, 'Submit');
Read more
catch's picture

SimpleTest handbook pages - lots of updates to do

So in the process of writing my first test I found a bunch of documentation discrepancies at http://drupal.org/simpletest

I'm not yet familiar enough with everything to know exactly what's deprecated, what's best practices/optional etc. so starting this to bring them up - then we can just edit stuff directly on the page once it's confirmed it needs changing. Or if they're not salvageable, just archive them quick so we're not presenting misleading information and replace with http://groups.drupal.org/node/11020 when it's ready.

Read more
Souvent22's picture

Selenium and Drupal

First, for those that don't want to read the full post, here's the "30-second elevator speech":
This post to to discuss using Seleinum with Drupal. Specifically using Maven to run the selenium tests and writing the Selenium tests in Java.
The example code be downloaded from the Workhabit Inc. public repository here: https://svn.workhabit.com/svn/public/drupal/selenium/trunk
One must have the following installed to run:
1. Firefox
2. Java 1.5+
3. Maven
All 3 are easy to install and aquire. Once you download the code from the repository, you can just navigate to the directory and run:

Read more

My first SimpleTest

Having not written a single test yet I figured it'd be a good idea to document the process, and I happen to have an issue in the queue which is perfect for doing so.

Background:
* Access rules were removed from core
* Statistics module had a direct call to the {access} table which I knew nothing bout, so is now broken
* Dries said "We should probably write a test for it too so we don't reintroduce this problem."

Read more
Subscribe with RSS Syndicate content

Testing and Quality Assurance

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: