Modernizing Testbot: Drupalcon Austin Update

Background:

"Modernizing Testbot" is a Drupal community initiative with the goal of rebuilding Drupal.org's Automated Testing / Continuous Integration infrastructure, leveraging modern practices and tools in order to enhance the capabilities and functionality provided by the platform and better meet the changing needs of the Drupal community. The initiative first took root with the vetting of a potential design and architecture during the DevOps Summit at BADCamp 2013, which then led to the development of a functional Proof of Concept during DevDays Szeged. DrupalCon Austin saw a number of further refinements to the model, and launch of the official project pages/repositories on Drupal.org. This post is intended to provide a snapshot of what was accomplished.

Drupal.org Project Pages

The week before DrupalCon saw the launch of the 'DrupalCI' project on Drupal.org, which is intended as the umbrella project for related 'Modernizing Testbot' work. The initiative has been broken down into five sub-projects under the same DrupalCI namespace. The initiative is still young, and code in the following repositories is not necessarily ready for use, but this is where you will find things during development and after the platform goes live:

1. drupalci_drupalorg: Responsible for integration between drupal.org and the DrupalCI platform
2. drupalci_jenkins: Responsible for the scripts and configuration files used on the jenkins master/slave servers which serve as the central job dispatcher for the environment
3. drupalci_testbot: Responsible for the job runners performing the actual work, but also functional as a standalone project for local testing
4. drupalci_results: Responsible for the long-term archival, storage, and exposure of build artifacts and job results, similar to qa.d.o today
5. drupalci_puppet: Responsible for the puppet configuration scripts used to build out the various servers which make up the environment.

Development Infrastructure

Also in the weeks leading up to DrupalCon Austin, we set up a full development environment with the help of the Drupal.org Infrastructure Team. This environment has been set up entirely on Drupal.org infrastructure, and is intended to closely mimic that which would be used for a production launch of the platform. A special thanks goes out to nick_shuch (PreviousNext) for his help in setting up the jenkins server and initial drupalci_puppet scripts.

Multiple Environment Support

One of the primary testbot sprint goals for the week was to enhance the proof of concept build, enabling support for testing on multiple database types and php versions. In addition to his contributions as sponsor of the DrupalCon sprints, user jaredsmith (Bluehost) also joined the testbot sprint team; where he started with developing a Docker build which would add support for a Postgres 9.1 test environment to the platform ... and then followed that up with builds for MariaDB 5.5 and 10 as well. With these Docker builds now available in the repository, we are ready to begin integration and testing of these environments with the actual run scripts; and any assistance the community could provide with this task would certainly be welcome.

Initial Drupal.org Integration

While it will be quite some time before the DrupalCI platform is actually integrated with Drupal.org itself, we anticipate the need for some means of triggering tests from a Drupal.org dev site, in order to demonstrate the full end-to-end communications path as envisioned for the final deployment. User Psikik dropped by our table during the Friday sprints in order to express an interest in the project, and looking for some way to chip in. Despite admitting to have never built a drupal module before, Psikik took on the task of developing the initial drupalci_drupalorg implementation, providing the ability to trigger jobs on our remote Jenkins server from a form located on Drupal.org (or, in this case, a D.o dev instance); with his final result ending up committed directly as-is (other than namespace changes) to the drupalci_drupalorg repository.

Test Runner Refinements

Throughout the week, and especially during the Friday/extended sprints, users dasrecht and ricardoamaro (original author of the Szeged proof of concept, sprinting remotely from Portugal) made a number of refinements to the test runner bash scripts; including refactoring to a new directory structure which should better position the project for an expanded set of functionality in the near future. Dasrecht also contributed an initial framework for a future 'drupalci' console command, which will eventually serve as the primary end-user interface for local interaction with the platform.

Jenkins scripts

Also during the Friday sprints, user geerlinguy and I both made progress on the development of jenkins scripts which will be triggered in order to kick off new jobs/test runs.

Where to from here ... What's next?

Starting next week, I intend on establishing a rotating weekly "Modernizing Testbot" call; alternating each week between North/South American and Europe/Asia/Australia-friendly timezones. Progress and updates on the initiative will be shared via the 'Drupal.org Testing Infrastructure' group on groups.drupal.org, so anyone interested in the ongoing progress of the initiative is encouraged to sign up there. Our immediate priority will be further testing and stablizing the Postgres and MariaDB environment support for local testing purposes, after which we'll turn attention towards the investigation of issue queue/patch workflow improvements which could be supported by the platform.

As always, testing and contributions are welcome; if you would like to get involved, please feel free to contact me via my Drupal.org contact form.

Bonus Points: QA.Drupal.Org "Test Complete" Notifications

While not directly related to the 'Modernizing Testbot' activities, I simply cannot leave without a huge shout out to justafish, who approached me during the extended sprints, asking what it would take to enable json support for qa.d.o test results. After I provided her with an admittedly not-so-helpful answer (i.e. "If someone else was to do the work, I'll deploy it; but it otherwise isn't a priority") and a qa.d.o dev site, she had a working json feed, PIFR patch, and new view definitions completed in no time. This work was deployed live to qa.d.o on Tuesday, so that end users can now access the json view of their test by appending '/json' to the 'view test' URL. This in turn enabled development this week of a chrome extension (also by justafish), which can monitor your tests in progress and notify you as soon as the qa.d.o test run is complete ... a project that I anticipate will very quickly find it's way into the toolset of Drupal core developers everywhere.