DrupalCI MVP walkthrough

The Drupal Association staff just gave a demo to a few of the core committers of the new MVP of the DrupalCI project. Here are some screenshots I took along the way to show how it will work!

(Note: Some of these details may change prior to deployment. This is a snapshot in time.)

Multiple environment support

DrupalCI supports testing across multiple environments (PHP versions + databases). It adds a new UI to the "Automated testing" tab of projects that looks like this:

This allows project maintainers to specify for each environment what behaviour they want:

• Test the branch both on (needs review/RTBC) issues + on commit (we'd want this for our "primary" environment, which atm is PHP 5.4+MySQL 5.5 for Drupal 8).
• Test the branch only on commit (we'd want this for any "secondary" environments we know are already passing, such as SQLite).
• Test the branch daily. (We'd want this for any environments we know have open fails, such as PostreSQL)

Below, you can see the list of current results for each environment at the branch level. This is also where you can trigger re-tests if you're a committer:

Running (and re-running) tests from the issue queue

For any environment marked "Issues & on commit", any patches will automatically queue against that environment:

In addition, you can click the "add tests" link to also manually queue up additional environments for a given patch, for example if a patch is suspected to impact PHP7 or PostgreSQL:

This is also the UI you use when you want to re-test a given branch/environment combo (once testing is complete).

Viewing test results

While eventually we will show test results in a prettier UI similar to what we have now on qa.drupal.org, for now test results are viewable directly in Jenkins. Here's a sample test run task, with no failures.

Test results are output in JUnit XML, and you can click to "drill down" for more details if you want:

In the event a test run fails, the failures are visible immediately on the task page with no clicking required. You can click to expand each row for more details:

What I especially like is you can review the output of run-tests.sh in real-time as it's happening by clicking into the console output of the Jenkins task:

Automatic re-queuing

One of the exciting things about DrupalCI is it requires far less manual intervention. The "Naginator" can spin up testbots on demand, as well as automatically re-queue tests. This means far less latency in the event of a testbot meltdown and far less lead of manual human intervention when e.g. sprints happen:

What's left to do?

There are a few areas where there are holes compared to current functionality, but they don't block initial (parallel) deployment to production:

• E-mail notifications on environment failures, and a UI for people to sign up for said notifications.
• Sort out access control for a limited subset of people to cancel tests from Jenkins.
• Some other things I forgot. :) See https://www.drupal.org/project/issues/drupalci for more details.

When can I use this awesome stuff?!

The plan is for the DrupalCI MVP to get rolled out in parallel with PIFT/PIFR (the old testbot) in the next week or two, for core only to start, so we can see both sets of results and troubleshoot/improve further (for example, better integration of results into Drupal.org itself).

The hope is that said "burn-in" period goes well, and we'd turn off the old testbot about a month or so after that.

WOOHOO! HUGE thanks to everyone who's contributed to this fantastic project, and particular shout-out to Jeremy Thorson and Nick Shuch, whose contributions were especially critical.