Drupal.org Testing Infrastructure

This project is intended as a collection and organization point for the 'Modernizing Testbot' project, an unofficial Drupal community initiative with the goal of improving the Drupal.org automated testing environment. This is being accomplished through the design and implementation of a new automated testing / job dispatch infrastructure to replace the existing PIFT/PIFR module combination currently being used; increasing both the flexibility and functionality of the current environment in an attempt to bring it up to speed with other modern continuous integration platforms and better meet the evolved testing needs of the Drupal community.

Shyamala's picture

Modernizing TestBot Initiative meeting #9 2015

Start: 
2015-03-04 02:30 - 03:00 UTC
Organizers: 

The Testbot team is conducting public meetings. Want to join & help them, need support or just talk to the team, join the meetings! Come and share what you did last week & the plan for the weeks ahead.

For those of you who can't, please watch out we will be coming with more Europe friendly time slots shortly.

To Know more about the testbot initiative check out Wednesday,March 4th, 2015: 2:30AM UTC

Read more
hestenet's picture

Modernizing TestBot Initiative meeting #8 2015

Start: 
2015-02-25 02:30 - 03:30 UTC

The Testbot team is conducting public meetings. Want to join & help them, need support or just talk to the team, join the meetings! Come and share what you did last week & the plan for the weeks ahead.

For those of you who can't, please watch out we will be coming with more Europe friendly time slots shortly.

To Know more about the testbot initiative check out this blog post.

Read more
Shyamala's picture

Modernizing TestBot Initiative meeting #7 2015

Start: 
2015-02-18 02:30 - 03:00 UTC

The Testbot team is conducting public meetings. Want to join & help them, need support or just talk to the team, join the meetings! Come and share what you did last week & the plan for the weeks ahead.

For those of you who can't, please watch out we will be coming with more Europe friendly timesolts shortly.

To Know more about the testbot initiative checkout: https://www.previousnext.com.au/blog/architecting-drupalci-drupalcon-ams...

Read more
Shyamala's picture

Modernizing TestBot Initiative meeting #6 2015

Start: 
2015-02-11 02:30 - 03:30 UTC
Organizers: 

The Testbot team is conducting public meetings. Want to join & help them, need support or just talk to the team, join the meetings! Come and share what you did last week & the plan for the weeks ahead.

To Know more about the testbot initiative checkout: https://www.previousnext.com.au/blog/architecting-drupalci-drupalcon-ams...

The meeting will be held as a Google+ Hangout (online video) at: https://plus.google.com/hangouts/_/calendar/cmFqYXJhbS5zaHlhbWFsYUBnbWFp...

Wednesday, Feb 11th, 2015: 2:30AM UTC

Read more
Shyamala's picture

Modernizing TestBot Initiative meeting #5 Jan 2015

Start: 
2015-02-04 02:30 - 03:00 UTC
Organizers: 

The Testbot team is conducting public meetings. Want to join & help them, need support or just talk to the team, join the meetings! Come and share what you did last week & the plan for the weeks ahead.

To Know more about the testbot initiative checkout: https://www.previousnext.com.au/blog/architecting-drupalci-drupalcon-ams...

The meeting will be held as a Google+ Hangout (online video) at: https://plus.google.com/hangouts/_/calendar/cmFqYXJhbS5zaHlhbWFsYUBnbWFp...

Wednesday, Feb 4th, 2015: 2:30AM UTC

Read more
Shyamala's picture

Modernizing TestBot Initiative meeting #4 Jan 2015

Start: 
2015-01-28 02:30 - 03:00 UTC
Organizers: 

The Testbot team is conducting public meetings. Want to join & help them, need support or just talk to the team, join the meetings! Come and share what you did last week & the plan for the weeks ahead.

To Know more about the testbot initiative checkout: https://www.previousnext.com.au/blog/architecting-drupalci-drupalcon-ams...

The meeting will be held as a Google+ Hangout (online video) at: https://plus.google.com/hangouts/_/calendar/cmFqYXJhbS5zaHlhbWFsYUBnbWFp...

Wednesday, Jan 28th, 2015: 2:30AM UTC

Read more

Documentation for DrupalCI APIs - [Draft]

This is a work in progress, please feel free to add or modify.

Overview of DrupalCI API Project

This sub-project provides an API layer between Drupal.org and the DrupalCI dispatcher, allowing the modular substitution of backend components without requiring code changes within the Drupal.org integration module.
Link to Project: https://www.drupal.org/project/drupalci_api
Link to the issue queue: https://www.drupal.org/node/2373627

APIs

  • jobRun: Runs a Jenkins Job, to be called by the DrupalCI module to automate the job run
Read more
nick_schuch's picture

DrupalCI results component

One component which had not received much attention until the DrupalCon Amsterdam sprints is the results server. This is responsible for archiving and exposing test results and build artefacts for all DrupalCI jobs. Im happy to announce that this component is now implemented as a minimal viable product (MVP). So what does MVP mean? It means we now need to have a serious discussion about how the project should progress from here.

Here is a demo on the component in action!

https://www.youtube.com/watch?v=QYj7_bDnKj4

Read more
nick_schuch's picture

DrupalCI weekly meeting #3

Outcomes

  • Results CLI generating a "Summary" string based on Junit and Drupal XML artefacts.
  • Lots of work done on documenting infrastructure and how to access environments.
  • We can now trigger a build through the API, this results in a new testbot and build.
  • Currently looking at moving the weekly meeting to a more desirable time. Please contact Jeremy, Ricardo or myself if you wish to join.
Read more
nick_schuch's picture

DrupalCI weekly meeting #3

Outcomes

  • Results CLI generating a "Summary" string based on Junit and Drupal XML artefacts.
  • Lots of work done on documenting infrastructure and how to access environments.
  • We can now trigger a build through the API, this results in a new testbot and build.
  • Currently looking at moving the weekly meeting to a more desirable time. Please contact Jeremy, Ricardo or myself if you wish to join.

Goals

Nick Schuch:
* Finish of Symfony console application for results site interaction.

Read more
jthorson's picture

DrupalCI: Drupal.org Integration - Status and To-Do's

A brief outline of the current status and To-Do's for this DrupalCI component can be found within the Google Document link below:

https://docs.google.com/document/d/1QM7AIW6Qjh413mecCgEv55QEbMA-ryf5HGiP...

Summarized version: We have a simple form which will allows us to trigger Jenkins jobs, but still need to build out the automated triggering, configuration pages, on-demand job triggering capabilities, and all of the associated User Interface designs for deployment on drupal.org.

Read more
nick_schuch's picture

DrupalCI weekly meeting #2

Notes

  • Results installation profile is finished and ready for a review.
  • Consider installing a clone of this Results site to act as a "Github GIST" style site eg. Push my artefacts (logs db etc) and share with others for 24 hours. This is long term though.
Read more
nick_schuch's picture

DrupalCI weekly meeting #1

Overview

DrupalCI will start posting weekly updates to keep the community informed on the progress of the initiative. This is the first, welcome!

The most notable items from this weeks meeting are:

  • Jeremy has an initial implementation of the "Simpletest" build step in the Job Runner.
  • Nick has some mockups for a minimal viable product of the Results site.

Attendees

  • Nick Schuch (nick_schuch)
  • Jeremy Thorson (jthorson)
Read more
jthorson's picture

Modernizing Testbot: Next Steps & (bi?)weekly calls.

Hey all!

First of all, I'd like to extend a great big thank-you to everyone who chipped in to work on the Modernizing Testbot initiative during the DrupalCon Austin sprints, both locally and via remote!

Read more
jthorson's picture

Modernizing Testbot: Drupalcon Austin Update

Background:

"Modernizing Testbot" is a Drupal community initiative with the goal of rebuilding Drupal.org's Automated Testing / Continuous Integration infrastructure, leveraging modern practices and tools in order to enhance the capabilities and functionality provided by the platform and better meet the changing needs of the Drupal community. The initiative first took root with the vetting of a potential design and architecture during the DevOps Summit at BADCamp 2013, which then led to the development of a functional Proof of Concept during DevDays Szeged. DrupalCon Austin saw a number of further refinements to the model, and launch of the official project pages/repositories on Drupal.org. This post is intended to provide a snapshot of what was accomplished.

Read more
jthorson's picture

Modernizing Testbot: Drupal Dev Days report

Well ... it's been a few days since Drupal Dev Days wrapped up, and my Hungary experience is now coming to an end. Before I go, however, there are just a few things I need to wrap up.

Read more
jthorson's picture

Modernizing Testbot: Introduction

A couple months ago, at the BADCamp devops summit, we held a workshop to discuss the architecture and operation of the current Drupal.org Automated Testing infrastructure, as well as to review a proposed future state architecture for the platform.

This post is an attempt to summarize the discussion, outline the proposed architecture as presented, and discuss some next steps.

Read more

Modernizing Testbot: Volunteers and Roles

Interested in participating in the 'Modernizing Testbot' initiative? Please add your name to the list below; and if there is a particular piece of the architecture that you'd like to focus on (or have previous experience with), please mention it as well!

Please edit this page to add your info below!

Read more
jthorson's picture

Want to help design the next iteration of Drupal's Automated Testing infrastructure?

On October 18th, 2007, Chad Phillips (hummonk) made an initial commit to the ‘Project Issue File Test’ module, with the description “initial commit of an integrated file testing platform for project issue module”. This kicked off the pursuit of automated testing integration between drupal.org and qa.d.o (then referred to as testing.drupal.org). In 2009, Jimmy Berry (boombatower) performed a significant overhaul and redesign of the system with the release of PIFT/PIFR 2.0, introducing a number of structural and architectural changes and improvements.

Read more
jthorson's picture

Qa.drupal.org Testing Metrics & Report

Recently, I was asked to pull together some metrics which would help illustrate the current health of the automated testing infrastructure, and provide a baseline by which we could monitor it over time ... and I thought I'd share some of those findings here.

The first graph shows how testbot utilization has grown over time, from a steady-state count of around 1000 tests per month in the first half of 2011, doubling to 2000 tests per month in January 2012, and reaching a peak of over 6400 requests in June of 2013.

Read more
Subscribe with RSS Syndicate content