Summary: 20 000 patches submitted by 1200 contributors for Drupal 7 will require:
* testing tools development with patch testing automation
* 1200 core contributors to be trained on how to write and maintain core tests
The tantalizing goal The Drupal project lead has presented an interesting challenge to the Drupal project's core development contributors. If contributors add 100% testing coverage to Drupal 7, they can reap the benefits of automated testing by having four more months of feature improvements in the development cycle. Dries has left the Drupal community drooling for more development time.
Current state
The problem with Drupal testing
The Drupal community has already begun to embrace testing for Drupal 7. Core contributors are beginning to provide additional tests, or improvements to existing tests along with their patches. If the Drupal 6 development cycle grew to 9425 patches submitted by 741 unique developers, then it's possible the Drupal 7 development cycle will have 20 000 patches with 1200 or more core contributors based on growth rates between previous releases. This presents two challenges for the Drupal community. First, test creation tools, patch testing automation, functional test creation, unit test creation, and maintenance of those tests against the HEAD branch present serious systems integration and collaboration challenges. Second, assuming this complex infrastructure is working flawlessly then we must go about training 1200 potential core contributors. Contributors will need to make a huge initial investment in test driven development and change the way they contribute to Drupal. This may takes years. We are going to need a full suite of development best practices and techniques to succeed.
The primarily method for developer communication is the project module that runs on Drupal.org. We had approximately 200 000 issues and follow-ups in 2007. Thanks to great maintenance by Derek Wright, Chad Phillips, Adam Light, and Jimmy Berry the project module should scale to handle significant growth. Our infrastructure team lead by Gerhard Killesreiter with lots of leadership by Narayan Newton and backed by reviews from Jeremy Andrews and David Strauss should allow the Drupal infrastructure to scale to meet increases in both patch submissions and testing.
Stephen Peck and the documentation team continue to make great progress on the huge challenge of gardening Drupal's documentation. They will continue to provide a solid framework for patching and test creation documentation to be created. The popular Drupal dojo is a great model for trainin. Charlie Gordon and as well as others will step up to bootstrap the Drupal communities test creation and test maintenance instruction with Drupal dojo. Don't get me wrong, none of these efforts are without their own great challenges, and we are relying on dozens of contributors to perform as admirably as they have in the past. The beating heart of Drupal is the 24/7 banter and friendly collaborative discussion that go into IRC channel #drupal, now one of half a dozen specialized Drupal channels. The #drupal IRC channel will continue to be the friendly introduction for person to person support, and test development training, the community does so well.
However, there one is technique the Drupal community has relied on to make great progress on in the past that we can use again. That's the Drupal developer sprint. We've done sprints at:
- Drupalcon Amsterdam 1 day bug-fix sprint for 4.7
- OSCMS Yahoo had a sprint
- St. Charles, Illinois lead to Forms API III
- At Drupalcon Barcelona, the CCK redesign efforts are proving to be successful
- The Data architecture sprint in Chicago led to Database TNG and the registry patches
A testing sprint could be critical technique for ensuring the complex infrastructure we are building is ready to support 100% test coverage.
Solution
We are currently proposing that we conduct a testing sprint with at least four key testing infrastructure developers at DrupalCamp Paris. The sprint offers a mid way point between testing contributors Karoly and Rok, who are both located in Eastern Europe and who recently traveled to Drupalcon Boston.
The sprint aims to accelerate progress across 5 different testing projects that will support the Drupal 7 testing initiative. In person communication, will allow for decisions and problems solving to occur at an accelerated rate.
Outcomes
- integrated plan for testing, including
- unit testing
- functional testing
- automated patch testing
- javascript testing plan
- improve automated test creation tools
- infrastructure testing plan
- coverage metrics
- find ways to improve consistency and quality of tests
- better understanding of current state of testing
- increased test coverage for critical areas; larger number and variety of tests
- infrastructure for accepting and integrating community developed tests
- improvements for http://testing.drupal.org
Deliverables
- high-level plan for testing
- standards / guidelines / best practices / templates
- significant number of new tests to increase coverage, focusing on key areas
- review / state of current tests report
- plan for and initial implementation for test submission and management process
Budget
There are many expenses with international travel and meeting. We've found one opportunity for testing sprinters to meet. DrupalCamp Paris is taking place April 19th. The organizers of DrupalCamp Paris, Ori and Louis from AF83, have been working with us to provide space, food, and partial accommodations.
- To bring 4 sprinters together for Drupalcamp Paris we estimate airfare for 4 people to be $3000 US. For 6 people, $5000.
- Hotel accommodation should be $400 for Friday, Saturday, and Sunday.
- Some meals will be provided by AF83, and the Drupalcamp Paris organizers
- Developers are volunteering their time. The developers are taking 1-2 days off work to travel to the event and volunteering for a full weekend.
Logistics
- Map of sprint
- Arrive the evening of the 18th in Paris. Depart Monday afternoon the 21st.
- List of potential hotels.
Friday arrival times by time of arrival from the wiki page:
Morning:
Charlie: Arrives Thursday morning, ~7-8 AM. Will likely be zonked at that time, though, as it will feel ~midnight. Staying with grandparents.
Douglas: Arrive Fri 825 AM. staying at Hotel Cyrnos, leave Monday 1:40PM
Kevin: Arrive Friday 11:05 AM CDG. staying at Hotel Cyrnos, leave Monday 12:55 PM CDG
Afternoon:
Jimmy: Arrive Friday 2:55 pm (local) staying at Hotel Acadia Opera, and will leave 12:50 pm (local) on Monday.
Evening, saturday
Rok: Arrive Friday 9pm CDG staying at Hotel Acadia Opera, leave Monday 9.20pm CDG
Karoly: Arrive Friday 9pm CDG staying at Hotel Acadia Opera, leave Monday 8:20pm CDG
Dries: 12pm saturday
Sprinters
- Karoli Negyesi - confirmed, funded by NowPublic, country Hungary
- Rok Žlender - confirmed, funded by NowPublic, country Slovenia
- Charlie Gordon - accomodation secured, Google sponsored, country USA
- Jimmy Berry - Confirmed, sponsored by Google, USA
- Kevin Bridges - Confirmed, funding by pingVision, USA
- Douglas Hubler - confirmed, funding by Acquia, USA
- Dries Buytaert - confirmed, funded by Acquia, Belgium