Proposal: Add a community testing period to each major Drupal release, before marking it as the recommended stable download.

damien_vancouver's picture

Problem Summary

Currently on, we recommend the very latest release version of Drupal core as the recommended download. But, it often takes Contrib and Docs much longer to catch up to the point where the system is stable and easy to use from an end user (site builder) perspective. Novice or less skilled site bulders who do not realize this and arrive "too early" get extremely frustrated as a result. Contrib developers are placed under extreme stress to have their modules ported as soon as possible, often long before they are ready to do so and with issue queues clogged by angry demands from frustrated users (instead of useful issue reports by more advanced users). Often by the time many users are able to work with the new release to build and upgrade sites, we are halfway through the next major release cycle. This makes it hugely expensive or impossible for many organizations to keep up. Finally, fixing bugs in the new release is very painful due to the necessity to freeze API changes and have strict backporting rules.

This problem causes a great deal of pain and conflict community-wide.

Proposed Solution Summary

This solution is based loosely on the very successful Debian Linux model. Please see the (brief) introduction to this approach at if you are not familiar with Debian stable/testing/unstable .

To implement this kind of "testing" release for Drupal would be laregly semantics and rethinking how we refer to releases on It would not require extra work or compromise from anybody, nor changes to existing workflow or versioning. In fact, it improves life and reduces stress for contributors.

We could do it with these (fairly painless) steps:

  • Starting with Drupal 8, we release each new major version of Drupal in an initial community-wide "testing" phase.

  • During this testing phase, the recommended download version on remains the previous stable version.

  • The testing version is recommended to contrib and docs contributors, experienced Drupal developers, those wanting the latest and greatest technology, and those evaluating the new version as part of their upgrade plan.

  • We manage the expecations of users running testing by informing them that bugs are still being fixed and there may be API changes to fix major problems that come up. Also that docs are largely unavailable and we have to write them as we go. Also that the contrib modules they may have relied on last version may be different or completely unavailable.

  • We Considerably relax the backport rules for the testing release. We warn users things may change to fix major bugs, but still try and keep any API changes to a minimum. This will greatly reduce the pain of fixing bugs that come up after the release, as they inevitably do once people start enabling contrib modules. See for a discussion surrounding how to relax those backport rules. However relaxing the rules as things now stand causes problems - stable no longer means stable in that case. Having this testing phase allows us to have our cake and eat it too.

  • The new release moves from being the testing release to the stable release either when the the next major release comes out. OR when we determine the new release + contrib + docs are "stable". We might want to do that ahead of a new release coming out if the new release is held up and the old one really is stable and ready for the masses. This determination is for the end user's benefit, and thus can be based on the overall state of the entire Drupal project, including Core, Contrib, and Docs.

Note: I do not propose we extend the EOL date of Drupal 6. We should EOL it as planned as Drupal 8 is released. going forward, the release cycle is not changing length, and so neither will EOL cycles. No one will have to maintain stuff for longer with this scheme. We are just slowing down when we call a release stable, to give the community at large a chance to test too.

Benefits of this solution

  • No contributors have to change their workflow, release schedule, or do any extra work.

  • Fewer upset users angrily demanding working code when it's not possible to provide it, and not understanding why (both for contrib modules and core bugfixes). Those using the new release have been warned and hopefully are there in a spirit of collaboration and to help get the new release stable. This should result in less frustration and conflict.

  • Fewer upset users feeling that no proper documentation or help is available. Docs will have had time to catch up. Furthermore, external Blog sites, review sites, programming books, and so on will have a chance to check out the new release and post tutorials and reviews before the general public arrives en masse. This should result in less frustration and negative end user experience.

  • It will be easy to have a much less stressful and much more successful pledge system. For example, module developers could pledge to start on their port the day the new release comes out, or have the first -dev version up to evaluation... rather than to have it finished and stable on that day. In hindsight, that proved quite difficult for those without access to a time machine. This should result in less frustration and conflict.

  • Core developers and bug-fixers can make fixes more easily with the relaxed backport rules. We can more comfortably bend the rules when we think it is really really worth it (for example, for a realy important security or performance fix that would not qualify for a backport to a stable release). This should result in less frustration and conflict.

  • Companies already running a previous version of Drupal can now depend on having a good 12-18 months to test out the forthcoming stable version and evaluate their upgrade path. (This provides the time needed to lock down scope and budget for a future upgrade. If contrib takes 12 months to catch up, and your budgeting is annual... 18 months is just not long enough). This should result in less end-user frustration and abandonment of Drupal.

  • We foster the environment a new generation of Drupal contributors with the testing release. This will include the more tech savvy users out there. By holding back the novice users, but encouraging the collaboration of power users, we will reduce the signal-to-noise ratio on issue queues and get work done more quickly. This may just result a positive feedback loop of awesome. This should result in less frustration and a more more rewarding contribution experience for everybody.

  • In the short term, Drupal 7 gets a doubled lifespan as a stable release. Which is a good thing, as for site builders, it is only now just getting really awesome with tons of available contrib and lots of awesome and up to date docs. Less end-user frustration and more time for people to make the tricky D6-D7 upgrade without feeling they've already dropped a major vesrion behind. This should result in less frustration and less abandonment of Drupal. This is especially important given the giant spike in adoption of Drupal 6. These new community members are now facing first major version upgrade and the complicated issues that surround that.

Drawbacks / Issues with this solution

  • Contrib may progress slower than without this solution, as less people will be using the new release.

This is an obvious concern. However:
-- contrib module developers are the same people that will jump on new testing releases right away,
-- Issue queues will contain more informed bug reports and those making them will be more polite and helpful.
-- a testing phase provides an opportuity for them to port while it's still laregely stress-free (rather than later once the release hits stable and users start demanding/expecting. Which is wrong of them, yes, but that doesn't stop it happening or the maintainer frustration from happening).

  • There is actually work involved: changing accordingly and documenting this change and the new system everywhere. This is probably a fair bit of work. I volunteer of course, but it's very broad-ranging in scope and thus will involve work for others too.

I believe this would offer extreme value to us in terms of the amount of benefit for the amount of work required.

I believe hat there is very little to be lost by attempting this, and so much to gain.

Lastly, I believe that if we commit to and start on this soon, we have plenty of time to do a really good job of it by the time D8 is released, and by the time D8 is declared stable, it will be the most tested, most stable, most awesome Drupal environment ever, from everybody's perspective.


I'm not clear on how the

Barrett's picture

I'm not clear on how the Testing phase you're proposing differs from the post code-freeze phase of the current process. Is the difference just a semantic change and a relaxing of the rules as to what can change while in this state or is there more that I'm overlooking?

We did this with Drupal 7, but without notification

rfay's picture

Drupal 7 was still in "beta" and completely immature for at least 6 months after the "release". It finally caught up with the most critical bugs (or at least a lot of them) by the summertime last year. So in your proposal, it could have been the "testing release" during that period, or perhaps even up to about now.

I think this proposal might clarify that situation, and might be perfectly reasonable. You're essentially codifying what the insiders know (and what's written in the handbook in various places): You have to see how mature all the piece-parts are before choosing a particular release.

So this is really a proposal about how we communicate maturity and recommended release to the rest of of the world.

I'd support it.

Thanks for your responses

damien_vancouver's picture

Thanks for your responses guys!

@Barrett asked

Is the difference just a semantic change and a relaxing of the rules as to what can change while in this state or is there more that I'm overlooking?

Yep, technically that is it..

@rfay summarized it well in his comment:

this is really a proposal about how we communicate maturity and recommended release to the rest of of the world.

That's the big idea in a nutshell. All of us reading this likely know these things already, and they are second nature. But novice users/site builders don't necessarily know this, and us recommending a .0 as "recommended" and "stable" is giving them the wrong impression and causing the issues listed above. Providing a formal, public "testing" phase will keep the right people (the less technical) at the old stable version until the new one + contrib + docs is ready for mass adoption without causing tons of support requests and/or negative impressions.

By making this semantic change, we manage expectations to reduce frustration, which then "trickles back up" to the core and contrib contributors. They're not overwhelmed with unreasonable support requests in the early release period when their time is much better spent elsewhere (on the new version, or on fixing/backporting issues reported by more tech-savvy and issue-queue savvy testing users). Being able to safely relax the backport/freeze rules for testing, without reducing stability of stable, is a side benefit of this proposal.

Lastly, from the perspective of an org evaluating an upgrade, they can now start doing this on a (mostly) stable new version once it enters testing. They then have the entire testing period (whether it is 6 months or an entire release cycle) to evaluate their upgrade, including the scary part - figuring out their upgrade path for contrib modules used in their previous version. Contrast that with how things work right now. The new recommended .0 release comes out after the core beta testing period is over, before any contrib modules are available. Beyond making sure core runs in a basic configuration that does not explode (which we already know thanks to simpletest and the core beta), organizations can't really test their real world use case or upgrade path. It's impossible to submit a complete budget request to management for a site upgrade without knowing these things. I've also heard several early D7 site build horror stories that begin with: "the customer was excited to use the latest Drupal release" and then carry on to a tale of woe about the project going 3 times over budget. My point there is the language we use to describe the release also goes beyond the implementors and site builders, to all levels of an organization, not just site builders.

Unofficial Drupal Roadmap

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds:

Hot content this week