After some discussions with my mentor Thomas and other people on devel mailling list I have created sort of a first prototype.
How it works
One part of this module is used to catch the patches and submites them to other part of the module via xml-rpc call. The second part receives this call and tests the patch. Results are stored in a new node on test server node address is returend by xml-rpc call response.
Catching pathces is done in currently done in two ways:
- With a block into which user writes url of the patch. Test result node address is displayed in drupal message
- With nodeapi hook that reacts if new project_issue with a patch attached is inserted. Test result node address is automatically appended to project_issue body. This currently only works for a new project issue follow ups are not monitored because when a new follow up is inserted nodeapi hook is not called.
When we have patch url xml-rpc call is used to transfer this url to test server. One of xml-rpc call parameters is also a password that makes this thing a bit more secure. This means you have to know which password is set on test server to make the xml-rpc call.
When test server receives xml-rpc call new node is immediately created, test script is started and it returns node url to first server in response to xml-rpc call.
Testing
Testing consists of two stages in first stage a new drupal installation is created and in second tests are run.
First stage
For the first stage I used slightly modified script that merlinofchaos submitted in one of his comments here http://drupal.org/node/52709.
Everything we need is checked out from cvs (drupal core, simpletest module and simpletest library) into a new directory named after this test instance name (test_12345678). Then database is setup. Every patch gets its own database again named test_12345678. Some initial data is inserted like admin user and also simpletest module is enabled with a direct insertion into system table. Next setting.php file is modified two things are changed database url and base url. Now we have a working drupal installation before testing it patch has to be applied. Currently patch only applies if it consists a line like this (Index: includes/common.inc) this can be improved but for now it is good. This now concludes first stage. Result of this stage is a working drupal installation with applied patch.
Second stage
Tests are run with modules/simpletest/run_all_tests.php script this script is called with wget and result is stored in a tmp file which is then read back and appended to node we created.
Known problems
- At the moment there are some tests that are broken and therefore it is not possible to really use this
- Only security that is used to prevent every user submitting patches into the system is perm hook
- I mentioned before problems with follow ups in other to fully automate this process this problem will have to be solved