Want to help design the next iteration of Drupal's Automated Testing infrastructure?

jthorson's picture

On October 18th, 2007, Chad Phillips (hummonk) made an initial commit to the ‘Project Issue File Test’ module, with the description “initial commit of an integrated file testing platform for project issue module”. This kicked off the pursuit of automated testing integration between drupal.org and qa.d.o (then referred to as testing.drupal.org). In 2009, Jimmy Berry (boombatower) performed a significant overhaul and redesign of the system with the release of PIFT/PIFR 2.0, introducing a number of structural and architectural changes and improvements.

By the time Drupal 8 is launched, we’ll have been running for 5 years on the current architecture, and almost 6½ years on the project. In that time it has served us well; but unfortunately the code has not kept up with technology and the changing needs of the Drupal community. The D8 launch brings with it a host of new requirements, such as an urgent need to test against PHP 5.4 and PHP 5.5; while simultaneously supporting PHP 5.3 for legacy D6/D7 contrib projects.

While we are committed to providing support for different PHP versions within the existing 6.x-2.x PIFR codebases (i.e. selected based on per core Drupal version), the ever-increasing demand for new features continues to stretch us beyond the design intent of the current system. Each new change brings with it the added risk of instability, at a time when stability is critical as the project drives towards a D8 beta release. As such, my preference would be to feature-freeze PIFR 6.x-2.x after the 'multiple PHP versions' requirement is delivered, and focus all future attention on the development of a new iteration of Drupal’s Continuous Integration tools.

I have a plan for that next iteration, one that leverages a blend of industry-standard 'Not-Invented-Here' tools with new modern approaches; providing a platform which will increase our testing capabilities and overall flexibility while decreasing our development cycle for new features and functionality ... all assembled in parallel to the existing environment in order to ensure the least possible disruption to the D8 development cycle. An outline of this plan was introduced at the Dev-Ops summit at BADCamp this fall, where it received general approval and an offering of support from numerous participants.

At that same event, I volunteered to help coordinate activities related to this initiative, and expect to be able to start formally structuring things sometime in the early January time frame. An overview of the entire plan will have to wait for a future post ... but in the meantime, think "PIFT -> Jenkins -> Docker -> PIFR", where the PIFR worker functionality is slowly augmented or replaced by specialized job workers over time, and the Jenkins server is potentially used as a generic job dispatcher for other future 'external' d.o features as needed.

Of course, this is an ambitious undertaking; and as such, it can only succeed through the support of volunteers within the community, people like you. To get involved, please take a moment to join the g.d.o group at https://groups.drupal.org/drupal-org-testing-infrastructure, and join the discussion below ... being sure to point out any relevant experience or specific area of interest that you'd like to help out in. Alternatively, feel free to email me via my drupal.org contact form (https://drupal.org/user/148199/contact).

I look forward to hearing your thoughts!


I can't wait for this initiative to begin! :-)

dsnopek's picture

I can't wait for this initiative to begin! :-)

Sounds great! I'm curious

Mile23's picture

Sounds great!

I'm curious about Docker and how it works. Will have to experiment with it.

Can we get started using

pwolanin's picture

Can we get started using Docker sooner? I opened an infra issue about it at https://drupal.org/node/2161175

It would seem like maintaining a Docker container that anyone wanting to donate machines could use would both improve consistency and greatly simplify deployment even with the current stack/versions.

This sounds great!!

webchick's picture

One small request: can we please rename the effing modules to "test server" and "test client" or something during this process? ;) And move them both under one project? (Testbot?)

Oh, and in terms of helping,

webchick's picture

Oh, and in terms of helping, count me in for helping with QA and docs. I won't be very useful on implementation, unfortunately.

Docker, Deis, Flynn

niccolox's picture

I have just started using the open PaaS built partly on Docker called Deis

and managed to get it running in local dev mode on Vagrant/Virtualbox
next I am working on getting Deis running in a KVM vm, unsure if thats practical

however, Deis has providers for Rackspace, Digital Ocean, AWS, baremetal and Vagrant

there are Docker images of Drupal already and although Deis is a 12 factor app factory/paas like Heroku, personally I can't see what is stopping Drupal working its way into the Docker ecosystem

Heroku founders invested in Pantheon and Pantheon did a talk on LXC containers / Docker at Badcamp
video not online, but there are other resources

an interesting article on Docker and Wordpress slumlording, which could be re-purposed for Drupal purposes is at Rackspace

Drupalcon presentation on Linux containers, Vagrant & Docker - Great Feedback

added Deis Docker does Drupal


aroushanzamir's picture

We can make this community totally synergistic and interesting! Lets move on this!!