SOC 2006: Unit test automation

Goal is to create a system that would be able to test every patch submitted into issue tracker.

CVS and other stuff can be found at http://drupal.org/node/66983

Drupal let the quality boat sail by

public
Robin Monks@dru... - Sat, 2008-03-08 16:24

Hold on, hear me out :o) We're a really focused and strong (and almost too commercial) community, and we really have put our focus on features and output, and we've lacked a lot when it's come to making it all work correctly in every possible case.

That's where quality testing comes in. I've been thinking about Drupal's quality assurance process for a couple years now, but in the last few months it's really come to a head for me. My personal "dam-breaking" moment came when the Batch API in Drupal 6 prevented install, and other tasks depending on the API from running automatically, or in text based browsers, or in browsers were people have been privacy conscious and disabled JS and Meta refresh tags.

Without proper quality trails this will continue to happen. And here's another point I'd like to make:

Continue reading and comment at DrupalFiles >

Drupal let the quality boat sail by

public
Robin Monks@dru... - Sat, 2008-03-08 16:24

Hold on, hear me out :o) We're a really focused and strong (and almost too commercial) community, and we really have put our focus on features and output, and we've lacked a lot when it's come to making it all work correctly in every possible case.

That's where quality testing comes in. I've been thinking about Drupal's quality assurance process for a couple years now, but in the last few months it's really come to a head for me. My personal "dam-breaking" moment came when the Batch API in Drupal 6 prevented install, and other tasks depending on the API from running automatically, or in text based browsers, or in browsers were people have been privacy conscious and disabled JS and Meta refresh tags.

Without proper quality trails this will continue to happen. And here's another point I'd like to make:

Continue reading and comment at DrupalFiles >

Volunteers needed for http://testing.drupal.org

Amazon's picture
public
Amazon - Thu, 2007-11-01 22:59

Hello, the Drupal software project is going to start using unit test automation for Drupal core. We are looking for volunteers, and people looking for full time work who are interested in this project.

  • Volunteer to lead the unit tests for Drupal 7
  • Volunteer to document installation of automated unit test set up for consultants and customers

Contact us via the contact form at: http://testing.drupal.org

Kieran


Volunteers needed for http://testing.drupal.org

Amazon's picture
public
Amazon - Thu, 2007-11-01 22:59

Hello, the Drupal software project is going to start using unit test automation for Drupal core. We are looking for volunteers, and people looking for full time work who are interested in this project.

  • Volunteer to lead the unit tests for Drupal 7
  • Volunteer to document installation of automated unit test set up for consultants and customers

Contact us via the contact form at: http://testing.drupal.org

Kieran


SoC 2007 proposal - Drupal Automated Staging Kit

Allister Beharry's picture
public
Allister Beharry - Fri, 2007-03-23 17:33

Hello all,
Sorry for cross-posting; but I think people working on different aspects of Drupal may have interest in this idea.

I've submitted a SoC 2007 Drupal proposal on automating the whole process of creating a complete Drupal site with a LAMP stack in a self-contained virtual machine image. A formatted PDF version is here:
http://www.abeharry.info/SoC2007_DrupalAST_FullProposal.pdf
Here is the abstract:

The Drupal automated staging toolkit is a proposed set of code libraries, file schemas and parsers, and code generators, for


SoC 2007 proposal - Drupal Automated Staging Kit

Allister Beharry's picture
public
Allister Beharry - Fri, 2007-03-23 17:33

Hello all,
Sorry for cross-posting; but I think people working on different aspects of Drupal may have interest in this idea.

I've submitted a SoC 2007 Drupal proposal on automating the whole process of creating a complete Drupal site with a LAMP stack in a self-contained virtual machine image. A formatted PDF version is here:
http://www.abeharry.info/SoC2007_DrupalAST_FullProposal.pdf
Here is the abstract:

The Drupal automated staging toolkit is a proposed set of code libraries, file schemas and parsers, and code generators, for


Thanks for the tour, Rok!

robertDouglass's picture
public
robertDouglass - Sat, 2006-09-02 20:22

With Rok's guidance, I tested out the automated patch testing system that he's created. I uploaded a number of patches to his site, which then applied them to either Drupal 4.7 or Drupal CVS, and then ran 16 unit tests against them. We tried patches that applied and contained no errors, patches that didn't apply, and patches that had coding errors. Each time the test suite performed beautifully.

The value of this work is that patches that linger in the patch queue often fall out of sync with a moving HEAD. At the point when that happens, Rok's tests would be able to detect that and notify the patch maintainer. Furthermore, Dries, Neil, Gerhard and Steven would never have to write "patch doesn't apply" (after just having wasted their precious time trying to apply it). Furthermore, people often submit patches without thinking through all of the ramifications that a patch can have. If we were to build up a thorough set of unit tests (I can't imagine 16 tests really cover Drupal), then a lot of these errors would be detected automatically.


Thanks for the tour, Rok!

robertDouglass's picture
public
robertDouglass - Sat, 2006-09-02 20:22

With Rok's guidance, I tested out the automated patch testing system that he's created. I uploaded a number of patches to his site, which then applied them to either Drupal 4.7 or Drupal CVS, and then ran 16 unit tests against them. We tried patches that applied and contained no errors, patches that didn't apply, and patches that had coding errors. Each time the test suite performed beautifully.

The value of this work is that patches that linger in the patch queue often fall out of sync with a moving HEAD. At the point when that happens, Rok's tests would be able to detect that and notify the patch maintainer. Furthermore, Dries, Neil, Gerhard and Steven would never have to write "patch doesn't apply" (after just having wasted their precious time trying to apply it). Furthermore, people often submit patches without thinking through all of the ramifications that a patch can have. If we were to build up a thorough set of unit tests (I can't imagine 16 tests really cover Drupal), then a lot of these errors would be detected automatically.


Simpletestauto module installation instrucitions

public
Rok Žlender@dru... - Tue, 2006-08-15 23:44

Here you will find all the information you need to setup an environment where you will be able to automatically test patches.

Simpletestauto module installation instrucitions

public
Rok Žlender@dru... - Tue, 2006-08-15 23:44

Here you will find all the information you need to setup an environment where you will be able to automatically test patches.

Automatic testing security issues?

public
Rok Žlender@dru... - Tue, 2006-08-01 11:21

One of the biggest concerns about automatic testing is how to prevent potentially dangerous code to cause harm to the server and other tests.

Automatic testing security issues?

public
Rok Žlender@dru... - Tue, 2006-08-01 11:21

One of the biggest concerns about automatic testing is how to prevent potentially dangerous code to cause harm to the server and other tests.

Using test automation

public
Rok Žlender@dru... - Sat, 2006-07-08 13:05

Here are short instrucions on how to use this module.

Using test automation

public
Rok Žlender@dru... - Sat, 2006-07-08 13:05

Here are short instrucions on how to use this module.

Automation what has been done so far

public
Rok Žlender@dru... - Sat, 2006-07-08 12:41

After some discussions with my mentor Thomas and other people on devel mailling list I have created sort of a first prototype.

How it works

One part of this module is used to catch the patches and submites them to other part of the module via xml-rpc call. The second part receives this call and tests the patch. Results are stored in a new node on test server node address is returend by xml-rpc call response.

Automation what has been done so far

public
Rok Žlender@dru... - Sat, 2006-07-08 12:41

After some discussions with my mentor Thomas and other people on devel mailling list I have created sort of a first prototype.

How it works

One part of this module is used to catch the patches and submites them to other part of the module via xml-rpc call. The second part receives this call and tests the patch. Results are stored in a new node on test server node address is returend by xml-rpc call response.

Syndicate content