Drupal.org Testing Infrastructure

Improve and expand Drupal.org testing beyond its current borders:

  • Improve testing reliability and capability
  • Test on additional environments (pgsql, sqlite, Microsoft, Selenium)
  • Improve documentation and understanding of our code, infrastructure, and capabilities.
  • Make PIFR more deployable in environments outside D.O.
jthorson's picture

Modernizing Testbot: Drupal Dev Days report

Well ... it's been a few days since Drupal Dev Days wrapped up, and my Hungary experience is now coming to an end. Before I go, however, there are just a few things I need to wrap up.

Read more
jthorson's picture

Modernizing Testbot: Introduction

A couple months ago, at the BADCamp devops summit, we held a workshop to discuss the architecture and operation of the current Drupal.org Automated Testing infrastructure, as well as to review a proposed future state architecture for the platform.

This post is an attempt to summarize the discussion, outline the proposed architecture as presented, and discuss some next steps.

Read more

Modernizing Testbot: Volunteers and Roles

Interested in participating in the 'Modernizing Testbot' initiative? Please add your name to the list below; and if there is a particular piece of the architecture that you'd like to focus on (or have previous experience with), please mention it as well!

Please edit this page to add your info below!

Read more
jthorson's picture

Want to help design the next iteration of Drupal's Automated Testing infrastructure?

On October 18th, 2007, Chad Phillips (hummonk) made an initial commit to the ‘Project Issue File Test’ module, with the description “initial commit of an integrated file testing platform for project issue module”. This kicked off the pursuit of automated testing integration between drupal.org and qa.d.o (then referred to as testing.drupal.org). In 2009, Jimmy Berry (boombatower) performed a significant overhaul and redesign of the system with the release of PIFT/PIFR 2.0, introducing a number of structural and architectural changes and improvements.

Read more
jthorson's picture

Qa.drupal.org Testing Metrics & Report

Recently, I was asked to pull together some metrics which would help illustrate the current health of the automated testing infrastructure, and provide a baseline by which we could monitor it over time ... and I thought I'd share some of those findings here.

The first graph shows how testbot utilization has grown over time, from a steady-state count of around 1000 tests per month in the first half of 2011, doubling to 2000 tests per month in January 2012, and reaching a peak of over 6400 requests in June of 2013.

Read more
wwhurley's picture

Communication between Jenkins and Docker

Looks like you can attach to the docker container's streams and then echo them back out so Jenkins can record them. An example can be found at https://github.com/salsita/jenkins-docker-skeleton. Alternatively you can run docker logs or -- I think -- docker run -a to attach the stdout / stderr streams. Also some information on looking at the logs at http://0x74696d.com/posts/docker-logging/.

kimmel's picture

Introduction to automation for content checking

This message has been cross posted to Drupal.org Improvements, Documentation Team, and Drupal.org Testing Infrastructure.

tl;dr

I want to use automation to improve d.o content quality. I do not have access to do this and I have been unable to get someone with access interested enough to get this project moving.

The long version

My vision for improving the Drupal Documentation and drupal.org can be broken into three equally important ideas:

Read more

Testing Infra Evolution - Requirements

Purpose

This page is intended as a collection point for requirements for the next generation of Drupal.org's automated testing environment. Please feel free to update this page with your own thoughts and ideas, and help contribute to the requirements which will define the next (r)evolution of Drupal's testing infrastructure.

Read more
jthorson's picture

D7 Drupal.org Port & Testbot Evolution Strategy

DOCUMENT PURPOSE

The purpose of this document is to outline a proposed roadmap for Drupal testbot evolution, as discussed during the testbot evolution sprint at DrupalCon Denver on Friday, March 23rd, 2012. Participating in the discussion were Jeremy Thorson, Jimmy Berry, and Randy Fay. This proposal is also influenced by additional consideration and input provided by participants in the TestBot BoF held one day earlier.

Read more
jthorson's picture

Thoughts on Drupal Testbot Evolution

Note: This is a copy of a blog post original posted on March 20th, 2012; with the original located at http://jthorson.doesdrupal.com/thoughts-on-testbot-evolution ... it is being reposted here in the interest of consolidating the information on Drupal.org infrastructure, and to improve discoverability by any potentially interested parties.


Having spent the better half of the last year within the PIFT/PIFR issue queues, chasing the holy grail of sustained testbot stability, I find myself reaching a bit of a crossroads.

Read more
webchick's picture

Upgrade Drupal.org to Drupal 7 Sprint!

Start: 
2012-04-23 (All day) - 2012-04-27 (All day) America/Los_Angeles

The week of April 23 - April 27, the Drupal Association is sponsoring a sprint, hosted by the Oregon State University Open Source Lab, to upgrade Drupal.org from Drupal 6 to Drupal 7!

The sprint will be mainly focused on the following areas, and the following attendees will be there:

Read more
webchick's picture

Drupal.org Office Hours

Start: 
2012-04-09 11:00 - 12:00 America/Los_Angeles
Organizers: 

Come to Drupal.org office hours to share what you're working on and help us come up with the weekly drupal.org hit list!

Stop by #drupal-infrastructure from 11am to 12pm Pacific Time (18:00 - 19:00 UTC).

pillarsdotnet's picture

Move all core Drupal files into /core folder

Start: 
2011-11-01 (All day) Europe/London
Organizers: 

On November 1, 2011, all Drupal 8.x core files will be moved into a /core folder. This will break nearly every patch in the 8.x queue, requiring them at least to be re-rolled against the updated directory structure.

After that date, backporting 8.x patches to 7.x will no longer be trivial.

Dries announced in #22336-254:

Read more

London Automated Testing/Project Dependency Sprint Plan 26 Aug 2011

The Project Dependency module is deployed and enabled on drupal.org, with initial dependencies built! Now we have to get to the next step, deploying the PIFT changes that go with it.

IRC: #drupal-testing

Read more
rfay's picture

London Project Dependency Sprint Friday

I'd love to do a code sprint on testing Friday at Drupalcon London.

The most important outcome would be deployment of Project Dependency and the related PIFT, PIFR, and Project module changes.

What we need to do to accomplish this:

  1. Set up a test environment (PIFT, QA.scratch + testbot) demonstrating dependencies working.
  2. Get the new PIFR version deployed, assuming it can be done without a PIFT change.
  3. Get the deploy issue RTBC'd and perhaps deployed.
Read more
rfay's picture

New docs page, and updated project pages

I wanted to point possible contributors to a new low-level testing infrastructure doc page, http://drupal.org/automated-testing - Although this is incomplete and probably incorrect, it's a starting place for documenting our testing infrastructure. I invite you to improve it.

Automated Coder Reviews for New Project Applications - Planning/Development

This wiki page has been set up to continue the 'Automated Coder' discussion initially started at http://groups.drupal.org/node/155084#comment-517764.

Read more
rfay's picture

Outstanding Progress on Selenium/Simpletest Integration

I want to make sure that everybody in these groups knows about the fantastic simpletest + selenium integration work that ygerasimov has been doing. I just tested out his Selenium module (7.x) and it worked fine for me.

Here's what I did:

  1. Install Selenium module and start Selenium Server as described in the INSTALL.txt.
  2. Installed the Selenium Test Example sandbox for sample tests.
Read more
rfay's picture

Testbots on OSU-OSL Supercell

I posted about the testbots being on the OSUOSL Supercell over at http://randyfay.com/node/99.

-Randy

Testing Test System Code Before Deploying

One of the problems we've always had that delays deployment of PIFR and PIFT code is that it has been hard to test. But now we have a workable test environment, consisting of three parts:

Read more
Subscribe with RSS Syndicate content

Drupal.org Testing Infrastructure

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: