Proposal: Create a predictable schedule around Drupal 7 point releases

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
webchick's picture

Update: Here's the summary and conclusion of discussion.
Update 2: Changes made to the version info page: http://groups.drupal.org/node/150854#comment-508464.

Due to a variety of factors, there hasn't been a point release of Drupal 7 since its 7.0 release, roughly 5 months ago. There are a lot of good bug fixes (some critical/major) and performance improvements in the 7.x branch right now, and the lack of these making it out there for public consumption yet has hampered the ability for contributed module authors to port their modules to Drupal 7, and several other bad effects.

I'm planning to roll the 7.1 release this week, but obviously this won't include all of the fixes to all of the bugs we know about, since some are still in progress, and some don't exist yet. Dries suggested, and I think this is a great idea, that a really good way to help get things rolling along (at least for the initial few releases while we clean up some backlog) is to develop a consistent schedule around Drupal 7 point releases, so if a fix didn't make it into 7.1, there'd be a new target date for a fix making into 7.2 (and 7.3, and so on).

Based on my travel schedule for the next few months, I'd like to propose a new point release of Drupal 7 the last Wednesday of the month. So that would be:

May 25: Drupal 7.1
June 29: Drupal 7.2
July 27: Drupal 7.3
August 31: Drupal 7.4 (just after DrupalCon London, so hopefully we can get some nice fixes in then)
...and so on.

(Note: Actual version numbers might be different, since if there's a security release as well we'll actually increment the number by two: one for just the security fixes, and one for all bug fixes to-date + security fixes.)

We could revisit this policy after 6 months or something and see how it's going and whether we can/should slow it down or keep it going.

I ran this suggestion past the security team first, and didn't see any objections to it. It seems like a great idea to me. What are your folks' thoughts?

Comments

I love this. Some will

aspilicious's picture

I love this. Some will complain because they "need" to upgrade their sites more, but I think that they will see the benefits of this in the end.

No problem with today's easier update instructions

juan_g's picture

I think there is no problem for monthly updates, because UPGRADE.txt was recently improved to include instructions for minor version updates such as 7.0 -> 7.1 (much easier than major upgrades such as D6 -> D7).

These instructions have hopefully fixed the previous problem caused by lack of documentation about this simple procedure, with many people unnecessarily disabling and re-enabling a hundred interdependent modules per site for each minor update. It's not a WordPress style click-a-button update yet, but it's a great step to facilitate frequent updates.

bug-fix only

catch's picture

Also the big change here is that currently Drupal core point releases tend to happen when there is a security fix to release, otherwise not so much.

If we move to a monthly release schedule, there is more chance of having some bugfix-only releases, and people don't have to upgrade to those if they don't want to.

timeboxed

bertboerland's picture

this timebox approach would definitely help in operations for those managing multiple sites as well as in sales to know when to sell what service (upgrades). I like timeboxed approaches for that reason. However, I am not too sure if the developers like it, it is done when it is done is -even for point releases- still something to consider givien the fact that we do not know how many resources will work on what.

In 2006 I wrote about this, see http://willy.boerland.com/myblog/why_drupal_needs_a_roadmap (timeboxing part), also see http://drupal.org/node/43305

What would you need to make this happen?

--

bert boerland

It's done when it's done, but

Xano's picture

It's done when it's done, but we're talking about a pretty continuous stream of fixes here, so while a single fix may be done, the stream will probably never stop. It's predictable (in a good way) to schedule monthly (or two/three/etc-monthly) releases and only diverge from that if 1) there are security fixes 2) there are no fixes and there is nothing new to release.

agreed

catch's picture

If the rate of fixes slows down to the point where it's not doing a new release, I don't see any problems skipping a month or two when that happens - we could still do the next release last Wednesday of the month.

I also agree that "it's done when it's done" doesn't count so much for point releases of core. 7.1/2 is going to be shipped with ~20 known critical issues and ~300 major bugs. However there are dozens or hundreds of bug fixes that have been committed since 7.0 was released - some of these for critical and major issues that are hurting Drupal 7 sites right now, so it makes sense to get those fixes released to the user base this week then knuckle down on what's left.

Fantastic idea! I agree with

jerdavis's picture

Fantastic idea! I agree with the above comments largely. I also think this provides some extra motivation for folks to work through things they want to get in. Having the deadline might provide encouragement to get something wrapped up.

Monthly releases sounds

jbrown's picture

Monthly releases sounds great!

There's a mature patch for populating width and height attributes in image tags for styled images: http://drupal.org/node/1129642 Would be great to get it into 7.1. Someone needs to RTBC.

yeees

mike stewart's picture

yeees

--
mike stewart { twitter: @MediaDoneRight | IRC nick: mike stewart }

Also, just to point out...

webchick's picture

If there's some kind of terrible, 0-day, in-the-wild core exploit, we would break this schedule and make sure that users' sites were protected. Similar if we released a 7.1 with "oops, we completely broke taxonomy!" This is just the schedule we'd try to adhere to in the non-emergency case, which is most of them.

I am more of a relase often

brosseaub's picture

I am more of a relase often relase early type of guy but in the git days that might be a thing of the past. Go for it, try it on

I am more of a relase often

brosseaub's picture

We can always adjust down the road if it prove to be too coercive

Release blockers

sun's picture

A date-based release interval schedule will only work if you implement an exception to the rule right from the start: Release blockers (regressions) won't allow for a release, regardless of what date we currently have.

Daniel F. Kudwien
netzstrategen

Not sure...

webchick's picture

I've been operating on that principle (0 criticals), and now we're at the situation we are now, no bug fixes, no performance improvements, no nothing for 5 months. Which means we're not releasing any critical bug fixes while some critical bug fixes exist. This is terrible, and has led to a huge stagnation problem, plus a general distrust of D7 as a platform.

I could get behind that if they were new "release blockers," directly caused by changes committed since the last point release. The only ones I'm aware of atm are http://drupal.org/node/936844 and http://drupal.org/node/1089174. The way I'd handle that is reverting the patch that caused it before Wednesday if we can't get to an RTBC patch in time, then try it again for D7.N+1.

But I'm no longer a proponent of the perfect being the enemy of the good. We need to get these fixes out.

Depends

sun's picture

There are two different kinds of release blockers:

  1. Simple ones (like http://drupal.org/node/1089174), which pertain to functionality introduced in -dev, and which otherwise didn't hit the masses yet. Especially when it's about internal/low-level stuff only (as in this example), I surely agree with you that we can rollback the original patch that caused the regression (unless there's a RTBC patch to fix it, as in this example).
  2. Complex ones (like http://drupal.org/node/936844), which actually have been introduced in 7.N-2, and which should have prevented a release of 7.N-1 in the first place. Especially when end-user/third-party experience is affected, we should take them seriously. In this particular case, 7.N-dev contains a fix for the original bug, but it seems to have introduced a different one. Unfortunate, but happens.

Perhaps we can remedy release blocker situations by formulating a "one week grace period" prior to each scheduled date. I.e., stop committing and caring for other bugs/issues, only focus on criticals and release blockers.

Unrelated to release blockers, I'm not sure whether aiming for monthly point releases isn't a bit too heavy in terms of frequency.

3 years (at minimum) * 12 months * 2 releases per point release - (5 initial months * 2 rppr) = Drupal 7.62 in the end; not accounting for potential security releases in between.

I surely don't mean the cosmetic aspect. Rather the sign and potential interpretation of healthiness.

Daniel F. Kudwien
netzstrategen

Yeah, going forward I think a

webchick's picture

Yeah, going forward I think a "code freeze" (catch even suggested an RC release) a week before the real release makes a lot of sense to shift focus only to release blockers. Obviously, that won't work out this time, but we can certainly do that in the future.

I think monthly release schedule for D7 is about right for right now, in order to clear through some of the "major" backlog. But as D7 matures, I'm sure we can reduce this frequency quite a bit. Perhaps bi-monthly after DrupalCon London, quarterly starting in 2012, etc. What might make sense is to set a date like December 1, 2011 to re-evaluate this schedule and see if it still makes sense, and adjust it if needed. Then maybe set a similar date 6 months after that, etc. I'm open to ideas.

Regarding simple vs. complex things, to me they're both the same. This process only works if it stays predictable, apart from flamingly critical anomalies. There are lots of bugs we should've fixed before 7.0 released and didn't. Some of them we have, others we haven't. We can no longer afford to hold back things we've already fixed on things we haven't yet. But by creating a structure around it, we at least know the next target date for getting those horrible fixes in the next time.

It might be a good idea to

jbrown's picture

It might be a good idea to make a release branch a week before so commits can continue to be made on the main 7.x branch.

release candidates

catch's picture

Yes I would really like us to do release candidates a few days before the point release. Like sun says we could only allow critical issues, release blockers, regressions to be committed that week. So this would be 7.2-rc - I don't know if update status handles this for core, it definitely does for contrib though so hopefully it's no extra work apart from the tag and the release node.

If people know there is going to be an RC, then hopefully at least some sites will build this into their development workflow/release process - and it also gives a marker for any API changes/additions that contrib modules might be waiting on so they can release point releases of their own.

On frequency, it's easy enough to see how many commits were made to the 7.x branch between a tag and HEAD, so we can evaluate that each month and if the rate starts to slow down significantly look at reducing release frequency. I can't see that happening for at least 3-6 months though, when it does we can still try to have a predictable schedule.

I can see the motivation behind jbrown's idea of branching when there's a release candidate, but that might complicate things a bit - if we have a three week sprint + one week qa/hotfixes process - then even if you miss the deadline for the RC with any one patch (with critical issues and security an exception), you know there's a decent chance it'll get released five weeks later. Right now people don't know if their patches are going to be released in days or months even if they're already committed to the branch. This would also require a somewhat steady flow of commits during the three weeks, instead of dozens of commits one day before the release, which would only help in terms of stability.

On 'release blockers', I was also thinking we should try to get down to 0 criticals before point releases, but that is proving impossible (the overall number is going up rather than down despite some eventually being committed), and pwolanin set me straight that it is silly hold off releasing a point release that fixes 5-6 critical issues, just because there are other critical issues that aren't fixed yet. Something is better than nothing.

Don't think so to an actual RC...

webchick's picture

I think a number of people here have pointed out some pretty real objections to overwhelming people with Update Status module noise. Additionally, with the security team's current versioning policy, we'd actually need to do a 7.2-rc, not a 7.1-rc, or else completely confuse people even more than we already do. :(

So I think if we instead enforce (and widely announce) pre-release code freeze, that might be a way to get the benefits that you want without overwhelming or completely befuddling hapless end users.

Don't make updates annoying.

Mark Trapp's picture

A predictable release schedule is great, but if these monthly releases only contain a handful of fixes (some of which are likely to be minor), we're creating a sort of banner-blindness scenario with respect to keeping Drupal updated.

When a new release of Drupal comes out and you have Update enabled, Drupal tends to be really annoying. You get a notice about it in Status Report, Update Status complains about it, and an email notification gets sent for those who enabled that functionality.

That's all fine when an update is really important, but if the monthly release schedule is only including a half-dozen fixes because that's all of the issues that got vetted in the previous month, it becomes a nuisance. So rather than getting people to update to the latest version of Drupal when it comes out, I fear people will just disable Update and grab the latest version when they get around to it.

So if we're going to go to a predictable release schedule, we should be sure that the update is going to almost always include useful/major fixes or improvements. If a monthly window gives us that, that's fine, but given it's been nearly 6 months to get 7.1, it doesn't appear like that's going to be the case.

If I'm correct in this observation, I humbly suggest making it once every two months.

Doesn't yellow vs. red handle that?

webchick's picture

Update status will annoy you in two different ways:

  1. If it's just a higher version number, it'll tell you you have updates, but they'll be indicated with just a yellow "warning" level notice.
  2. If it's a security release, or the project/release has been revoked entirely, it'll scream bloody murder with FLAMING RED.

If there were only minor bug fixes and no security problems fixed, Update Status would do #1, and you really could get around to updating it whenever you wanted.

I guess I misunderstood the

Mark Trapp's picture

I guess I misunderstood the purpose of having a predictable update cycle: if there are fixes we want to deploy on their sites because they block contrib development, I would think we'd want people upgrading as soon as possible (at least initially).

But we only have two release priorities, like you mentioned:

  • Upgrade now because it's a security update and every day you don't, a kitten gets it
  • There's an upgrade: do the upgrade or not, I don't care.

To push the necessary fixes needed for contrib, it seems we need to re-designate (at least for the time being) the latter priority as:

  • There's an upgrade: you don't have to upgrade, but you really, really should. Really.

7.1 is likely to be a big one, but if the releases after that only contain a small number of fixes, it diminishes the impact of the release notifications when we really want people to upgrade (and it's not a security issue).

So let's say 7.2 has 5 bug fixes, most of which are not immediately apparent that they're useful. I might consider it not to be all that urgent to upgrade. Then 7.3 comes out, again a few bug fixes, none of which look at that urgent. 7.4 is more of the same.

So now 7.5 is released 4 months later and I'm still running 7.1 because every time I look at the upgrade for that month, it doesn't look like it's worth the hassle. Drupal's just going to notify me in another month anyway.

Compare that to getting a long list of bug fixes for 7.2 after a couple of months. Some time has passed, and a long list of fixes make it look like 7.2 is worth the time it takes to upgrade and test my stuff.

Ok, cool. I think I see.

webchick's picture

Does my reply to sun above help? http://groups.drupal.org/node/150854#comment-502389

Basically, for at least the next 6 months or so, I think we'll have absolutely no problem finding LOTS of bug fixes we can get into D7 point releases to justify the Update Status notification. But I'd be fine amending the proposal to include a "revisit this policy on date X" where we could reduce the frequency if it makes sense. I think as D7 matures, this monthly requirement will definitely lessen.

Thumbs up from me.

Mark Trapp's picture

Yeah, that definitely helps, especially if the next few releases are going to be substantial even on a monthly schedule. Awesome.

Slightly off-topic and bike-sheddy, but it might be helpful to say, in the release notes for a point release, "Barring any critical security issues, Drupal point releases will now be on a N-month schedule" to keep people apprised of when to expect a new version, especially when the frequency changes.

Yes

Jeff Burnz's picture

For me this makes a lot of sense - I can manage my issue queues more effectively and even plan sprints around these dates focusing on a particular set of pressing issues. As opposed to the ad hoc/when its ready approach this is great and allows us to employ better issue management methodologies. +1.

+1, build in some flexibility

davideads's picture

This makes a lot of sense to me. It would create better release and issue planning discipline, and, at least for me, provide a nice "image boost" in the eyes of my bosses, who have embraced Drupal for our project but are wary of Drupal's inconsistent release schedule and the uneven quality of contributed modules.

As Webchick said above, a monthly release schedule need not be set in stone -- obviously critical security fixes should be released based on the threat assessment and other factors. Similarly, I'd presume there would be times when you'd want to skip a month. If there are a few scattered, minor fixes, or if there's a big nasty blocker in a given month, skip that month's release with a notification on d.o., which would double as a lightweight way for people outside the core developer community to keep tabs on the core development process. I could imagine the release schedule "decaying" over time as less and less minor fixes are committed to D7 core, and updates are only needed every two-three months, or even less frequently.

A little flexibility would go a long way in address sun's concerns about reaching Drupal 7.62. But the assumptions of a monthly release schedule for the next year or two would provide a much better framework for making those decisions than the current approach.

This is awesome! I agree with

Josh The Geek's picture

This is awesome! I agree with this 100%.

I go by Josh The Geek.
Drupal.org
GitHub

Absolutely

webkenny's picture

This is a fantastic idea and, IMO, will make all of us work just a little harder to RTBC and get otherwise stagnant bugs fixed/features backported. There is a slight sense of catharsis at least for me when working on tough bugs as I don't "see" the release coming often. Great idea. I am also very much in favor of throttling this as D7 matures. Now all we need is a major release schedule! ;) (Come on, had to try. Heh)


Kenny Silanskas
Proud Member of the Acquia Support Team

+1 ^_^

BTMash's picture

Given the things outlined above and clarified in the comments, I completely agree with a point release schedule.

As long as there are reasons to do it

IceCreamYou's picture

It sounds like the reason you are suggesting predictable release dates is to get momentum behind certain critical issues that have stalled, and get fixes to existing critical issues out the door. That's admirable, and so releasing regularly and often for these two reasons is a good thing. However, if we find that nothing particularly urgent is going into a release, let's not release it just because "it's time," please. Don't forget that new releases are also associated with all the work of testing and upgrading, and I don't think anyone will complain if they don't have to do that when there's no compelling reason to do so.

In other words, I'm fine with saying "if there are still criticals on this date but we have committed fixes to other criticals, we will release anyway" -- but I think it's disingenuous to say "we will definitely release on this date regardless of whether we've fixed anything by then." So I think that assessment should be on a per-release basis, not every 6 months or whatever.

But small updates can be easier

Fabianx's picture

However don't forget that it can be easier to update often and finding problems on a released shedule can also make the planning for updates more consistent. (i. e. schedule the drupal point release for the last Friday of the month; test it for one week in dev, then apply to production sites)

Best Wishes,

Fabian

For non technical reasons, we

cpliakas's picture

For non technical reasons, we as a community really annoy potential enterprise and government Drupal adopters when we say things like "it's ready when it's ready". This is the first step towards adding some sort of predictability to release cycles, which will in turn help ease technical thought leaders in choosing Drupal. Great initiative, big thumbs up here.

+1 for predictability

Thomas_Zahreddin's picture

this will make it easier to schedule service times and manpower(!) for site administrators;
for developers and maintainers: they also can coordinate module releases with the release date of new core versions and can test against these new core versions.

And of course there can be good reasons to have some extra releases (e.g. security).

We had from February 2008 to December 2010 20 Versions for D 6.x; so a monthly frequency should be fine.

what i would like to see

j0rd's picture

First let me comment that I've never had a major issue when upgrading from one version of drupal core to the next.

What I have had issues with is coding my site and running into issues which have been committed in to dev but not a point release. Same is pretty much said for contrib modules.

What I would like to see is a RC release created right after a new point release comes out. Stable patches should get committed to this RC one at a time when they are ready. Each patch needs to be logged in the changelog. Those who want to be annoyed with the latest and greatest can subscribe to this branch. I think point releases can come out when there are security patches that need to get pushed. These security patches will get added to the RC, so most will only get notified to upgrade when they need to. By proxy they would get all the added fixes in the current RC branch.

Something similar needs to be done for modules. Dev branch is occasionally to scary, but I usually require a patch or two beyond the stable to fix some bugs. Immediate RC branches where stable & tested fixes are pushed could be a welcome blessing for people lime me who are usually on the cutting edge of development with sites.

PS. Im all for monthly release cycles though. Im usually running dev anyways, and this would fix that problem for me. Additionally I get paid to upgrade my clients sites...so its more hourly billing

I think this is a great plan

pwolanin's picture

I think this is a great plan for now. I imagine we'd spread the the "scheduled" release to e.g. 2 months and then 3 months over the next couple years, but in any case having a predictable cycle.

I think a benefit here also is the deadline effect which always encourages people to get their stuff in.

Yes!

Crell's picture

I fully support this plan, for all of the various good reasons stated above.

One note, late in the Drupal 6 cycle core the sec team started issuing twin releases, one that was 6.x+sec patch only, and on that was 6.x+sec patch+any other stuff that was committed. Would we continue to do the same thing for Drupal 7? How would that be impacted by this scheduled approach?

No change regarding security releases

webchick's picture

They would still work the same. If we fix one or more security vulnerabilities that month, then the month's releases would consist of both a 7.N (security fix only) and 7.N+1 (bug fix + security fix release).

The main difference would be that the security team now has a target deadline that can both help them to sprint on bugs, as well as communicate clearly to exploit reporters the timeline for a fix.

Yes Please

jbrauer's picture

In institutions using Drupal one of the big challenges is the predictability of releases for change management boards etc. This would help a lot.

@crell I think from the note about version numbers may actually be different that the idea is to continue with the two-releases for security releases...

More releases please! I've

star-szr's picture

More releases please! I've definitely grown a "general distrust of D7 as a platform" from having to patch the same bug in core on nearly every D7 site I've worked on to date. The bug in question has not yet been backported to D7 (I would do it if I knew how).

I'm not sure I like the idea of scheduled releases, but I'm a newbie so what do I know.

This is tied into how fast we

Dave Reid's picture

This is tied into how fast we can currently move D8 with regards to bugs. More D7 releases will only work with increased progress on fixing issues from D8 to D7 that are currently stagnant.

Senior Drupal Developer for Lullabot | www.davereid.net | @davereid

Summarizing + Conclusion?

webchick's picture

Great; thanks for all of the thoughts here, folks!

So in summary it sounds like:

  • Adding predictability to the release cycle of Drupal is generally regarded as a good thing, as it helps site maintainers, it helps service providers, and also helps provide motivation for working on bugs.
  • Concerns were raised, however, about 1) overwhelming end users with the sheer number of upgrades (monthly might be too fast), 2) ensuring no release blockers make it into the release, and 3) making sure D7 fixes make it in fast enough to justify monthly releases.
  • Some things to combat those concerns would be: 1) not necessarily needing to release EVERY month, only if sufficiently critical problems are solved, 2) having some period of "code freeze" before a release (a week was suggested, though I think ~3 days might be more doable), and 3) http://buytaert.net/fast-tracking-drupal-7-bug-fixes.
  • We also need to be able to retain flexibility to break this cycle for "emergency" reasons like a 0-day exploit, a "paper bag" release, etc.

So, given that, any objections if I add a handbook page (linked to from the "Drupal core" project page) which says more or less the following:


Drupal core release schedule

New stable releases of Drupal core are scheduled for the last Wednesday of each month. A new release will only come out on this date if it's deemed by the Drupal core maintainers to contain enough important bug fixes to warrant a release; if not, you're safe until next the month! :)

If a release is warranted, a "code freeze" will be called a few days prior to the release date, and the impending release will be announced via the @drupal Twitter account and Planet Drupal. This makes a great time to update to the latest code in the branch for your version of Drupal, and help make sure the code is solid!

Exceptions to this schedule may be made in case of security vulnerabilities being actively exploited in the wild, or severely critical bugs that prevent Drupal from being used.

--

Sound good?

I'm happy with it.

j0rd's picture

You need to make sure there's a better distinction in versioning between the Security release & Security + Bug Fix release.

The current method of releasing two versions at the same time isn't obvious enough for most people, unless it's well explained. Something like ODD versions, will only contain security fixes and EVEN will be security + bug fixes.

I'm still confused if we use this system though. Will Drupal 7.3 (security only) be based off Drupal 7.1 or Drupal 7.2? I would expect it would include the 7.2 fixes as well....and if this is the case...why are we only doing the "security only" releases, if they're just going to include the bug fixes a month later. Seems like a waste of effort, although I guess that does give the 7.2 branch a month to mature and resolve any problems.

Could someone elaborate on this for me, so I fully understand.

I totally agree. I don't get

sun's picture

I totally agree. I don't get the point of the current security point releases either. AFAICS, there are only two valid procedures to handle them:

  1. Branch 7.x off into a completely separate, official 7.x-security branch

    The only way to do security releases that always contain security fixes only, now and in the future.

  2. Stop creating security releases, only provide security patches

    Like in the past. When the procedures for creating a new core security release are triggered, we already know of the applied security changes anyway; creating a cumulative patch is a matter of minutes.

I'd strongly prefer 2), since 1) has a underlying kind of "recommendation" that it's safe for you to skip all the other changes/fixes. In the case of Drupal 7.2, this is absolutely not the case. E.g., if you only update to the security release 7.1, then your site will break big time as soon as you try to use a module that depends on the crucial user_save() API fix, or any other critical/major bug fixes that have been fixed for 7.2.

I'd say it's OK to provide these security changes separately for Drupal experts only, but our current separate-security-release-per-official-release policy makes it too easy for the average Joe to do the wrong thing.

Daniel F. Kudwien
netzstrategen

Senpai's picture

Stop creating security releases, only provide security patches

Sun, exactly! With the rise of D.O.G. (Drupal On Git -- http://drupal.org/project/dog) we will have the ability to provide core security patches to most users because they will have built their site inside of the Git ecosystem. Thus, the majority of users can elect to either apply the security patch using Git, or skip the security patch and upgrade their site to the next point release (again using Git).

For the site admins who either don't want to use Git, or can't for some reason due to corporate rules or hosting company rules or etc, they'll either have to simply update their site to the latest point release in order to receive the security fix, or figure out how to create and apply a patch using their own non-standardized methods. The latter is still viable because is possible, normal, and sysadmins already do it. :)


Joel Farris | my 'certified to rock' score
Transparatech
http://transparatech.com
619.717.2805

Security Releases

jbrauer's picture

@j0rd --

7.3 will include everything that is in 7.2 whether or not it is a security release. If a security release is necessary and we're currently on 7.2 that security release will be 7.3. In Drupal 6.x and earlier it used to be that we'd do a release that included security fixes and bug fixes in a single release. This caused a number of problems and tough situations. First sites wanting the latest security fixes were forced to either use a patch and run a patched version of core that didn't exactly match any known version... or adopt the bug fixes at the same time as security fixes. So even if it had been a while and there were a bunch of bug fixes that needed testing on their site it took longer to get all those bug-fixes tested along with the security fixes.

So releases might look like this in the future... (this is purely hypothetical and not based on anything certain):

7.2 -- current stable release
7.3 -- Bugfix release -- contains all of 7.2 plus bug fixes
7.4 -- Bugfix release -- contains all of 7.3 plus bug fixes
7.5 -- Security release -- contains all of 7.4 plus just the security fixes
7.6 -- Bugfix release (may be released just after 7.5) -- contains all of 7.5's security fixes and new bug-fixes

The idea here is that it would be easy for a site to quickly adopt 7.5 knowing it fixes just the security issues. Then over the next days/weeks the site can test and adopt 7.6 before the next release is made but with a schedule that allows adequate testing of the bugfixes in 7.6 without putting their production site in jeopardy if there is a (fortunately rare) bug that slips through.

Right, and note...

webchick's picture

This is only done this way because of the capabilities and external dependencies of the tools we have. Changing version numbering schemes affects Update Status module, affects Drush, affects the packaging system, affects localize.drupal.org, etc. etc.

I proposed http://drupal.org/node/1172214 as an alternative, but I'm not sure if it's viable.

@jbrauer I got it. While that

j0rd's picture

@jbrauer I got it.

While that makes sense in theory, in practice though, I think those who are worried about upgrading and breaking their sites would only upgrade to 7.1 in the sequence you mentioned and skip 7.2, 7.3, 7.4. Then when a new security release is done in 7.5, they're in the same situation of having to upgrade 5 versions with out adequate testing. In reality though, I suppose that's their problem.

Of course as @sun mentioned, that's where patches are handy. In the past when I was more in-experienced and hacking core/contrib, I'd manually apply security patches instead of upgrading to new versions.

@jbrauer, I get the point of having easily trackable versions though. That makes sense. I get the point, that that's currently how Drupal update system works.

I'm personally ok with the dual release, but it's going to need to be properly explained to in-experienced users. Much like @jbrauer has just done for me. This explanation should be added into the change log and version news post, so that users easily understand and to avoid confusion. Additionally this should be properly announced, so that everyone knows what's going on when dual versions are released.

Something like @webchick has mentioned with minor versions would be best for security releases as it makes a little more sense visually....but I think with a little additional communication work, you could probably avoid the minor revisions for now.

--

To be clear though, the thing I would ideally like is a Drupal "testing" branch, in which stable bug fixes from dev are constantly merged. This branch could be frozen one week before a point release. The benefit of this is people like me, who don't really host Drupal sites, but rather develop them for clients constantly could keep current with this branch and easily report back if we have problems. Reason I want this, is when I'm doing dev, I regularly run into problems with core and end up having to search issues queues & patch. Sometimes a small bug in core takes me hours to track down to the issue queue & patch. While I try and avoid the dev branch, I'd be willing to put testing on my sites and upgrade my install each morning if there's a new bug fix. I mentioned this before, but no body seemed interested in the idea. When I'm about to deliver the site to the client I'd probably leave it on testing until the next monthly release is ready...then upgrade them to a "stable".

If this is an informal git branch, I'd be happy with that...just let me know where it is.

major/minor

catch's picture

I suggested keeping major/minor reserved for security/bugfix on the security team list, and still think that's the least worst option we have for explaining this to people - especially without any infra changes to support it - it's just keeping a convention and some documentation. It's good to know it's not just me.

MongoDB does not do this, but it does alternate development and stable releases (1.5 - development, 1.6 - stable, 1.7 - development, 1.8 - stable) so there is precedence for using the odd/even number to mean something.

Part of the problem is that

pwolanin's picture

Part of the problem is that any system should, ideally, be the same for core and contrib. Otherwise, it's as more or more of a WTF as to why core numbers jump around, but contrib don't, and then if there is a contrib SA, the version number may be anything at all.

In that vein, I think we either stick with this simple system, or start fixing update module to support an additional release version format. Or investigate whether it does already! Looking at the code, it might be more of a release/packaging issue rather than an update module issue - it looks like it just decides if you are out of date by string equality:

<?php
     
// Figure out the status, based on what we've seen and the install type.
     
switch ($projects[$project]['install_type']) {
        case
'official':
          if (
$projects[$project]['existing_version'] === $projects[$project]['recommended'] || $projects[$project]['existing_version'] === $projects[$project]['latest_version']) {
           
$projects[$project]['status'] = UPDATE_CURRENT;
          }
          else {
           
$projects[$project]['status'] = UPDATE_NOT_CURRENT;
          }
          break;
?>

I agree that the security

Dries's picture

I agree that the security point releases are really confusing. Just last week two people approached me about it; both thought we had screwed up the 7.1 release and that we were forced to roll a 7.2 release directly after. I think we should revisit how we do or name security releases.

@Dries - see above - this is

pwolanin's picture

@Dries - see above - this is the best mechanism we have unless we are going to change a lot of infrastructure and figure out how to deal with the install base of update module. Should we go back to supplying just patches? That was an error-prone manual process that NO contrib modules did that also leaves the site unable to use update status any longer.

The process we used for core is the same process we have used over the last year for contrib modules including Views and webform. I don't think you should be put off by people who can't be bothered to read the release notes. Rather, we need to just work a little more to educate people on how this works.

I think slowly changing the

Fabianx's picture

I think slowly changing the infrastructure to allow. 7.1-security1, 7.1-security2 is a good thing.

However I personally would just set into stone once and for all:

7.1, 7.3, 7.5, 7.7, 7.9, 7.11, ... = security fixes
7.2, 7.4, 7.6, 7.8, 7.10, 7.12, ... = normal releases

odd = security
even = normal release

If there are two security fixes needed after another, just do:

7.3, 7.5

The next normal release then is 7.6. The release after is 7.8, etc.

And maybe write into code, if update status == security update needed, then release has to be odd, and if not release has to be even.
(so that it gets enforced by the release infrastructure)

Yes, version numbers will increase faster, but at least, you would always know what it is.

Best Wishes,

Fabian

Ok, I'm changing my mind per

pwolanin's picture

Ok, I'm changing my mind per above comment to catch about update module and IRC discussion.

Apparently d.o will already potentially build some oddly named releases, but has some block from showing them on the project page.

e.g. catch made a 6.x-1.9-rc1 "release" of memcache module that shows up in the history and has a tarball, but according to him did not show on the project page:

http://updates.drupal.org/release-history/memcache/6.x/

So, enabling e.g. 7.1-security or some other new format for releases might be as "easy" as tweaking some of the project module and releated code, which is much easier than trying to fix the installed base of update. Obviously, we'll need to do some testing w/ fake release XML to verify that such a security release is handled correctly by update module.

Note that I am strongly opposed to an odd/even numbering convention unless we also impose it in contrib, which I think would be a nightmare. Hence, I'm strongly opposed in general.

Looks great

Dries's picture

Good morning, Angie et al. The proposed Drupal core release schedule sounds great to me. In fact, it is exactly what I had in mind.

I'd love to see us adopt this across all Drupal core versions. We should talk to Gabor to see if he wants to adopt the same release schedule for Drupal 6. I'm guessing he'd be on board with that.

self-update

Hadi Farnoud's picture

on completely unrelated note, security releases could distribute faster if we had sth like Wordpress automatic update.

Harder than it sounds

Crell's picture

The trick is doing that in a secure way (ironically). It took a lot of work to get the current module update/install process working securely, and it only works if your web host has the necessary tools enabled for PHP. (Apache writing code to disk is a Bad Thing(tm).)

Alternatively, I believe Drush now has core-update capability, which is 10x easier for anyone that has Drush installed, both for the code and for the user. Granted that doesn't help the run of the mill shared host, but perhaps we should incorporate bits of Drush into core for D8. :-)

I do agree that apache

Fabianx's picture

I do agree that apache writing files is a bad thing, but wordpress has a nice workaround for this:

Just insert your FTP data and it'll download the module and then you can enable it.

I don't think that is too bad from a security standpoint.

And there is not sooo much of a difference if the user installs it via FTP or the program. Just add digital signing to the mix (sign the tarball) and you are as secure as you can get.

Of course is drush a much easier way, but especially end users building sites won't necessarily have it.

Best Wishes,

Fabian

I've personally never been a

j0rd's picture

I've personally never been a fan of the WordPress way to update. Its very insecure and leads to poeople 777 their entire wp-content directory. This also leads to WordPress modules writing files under their module directories, since they base it off this assumption.

Personally with WordPress I only make the wp-content/upload folder writable, modify any plugin that don't write to this directory. I also update my sites by hand and don't use the web interface.

Having a non-ssl FTP on your server is a security risk, as the passwords are sent in plain text.

For non tech savvy users who don't have SSH though, I suppose they will be accessing their sites via FTP anyways, so I guess it makes sense to allow them to upgrade over an insecure connection rather than leaving their sites open to vulnerabilities. If they had to do it manually it would still be insecure.

Ill never support any initiative to give Apache more write permissions than it absolutely requires.

I do agree on the security

Fabianx's picture

I do agree on the security implications, but this way means that the less they use FTP, the more secure it is.

So if Drupal / WP / whatever allows update via FTP, the FTP password is then not sent in plain text as it just connects to localhost (in most cases) or at least keeps it on the internal network.

However then the question remains how to transfer the FTP password to the site and that of course then needs SSL again ...

But I guess even without FTP access, you are able to do already much things with a Drupal Admin password (given that you can enable PHP Filter Module ...)

So if we wanted to have this secure, we would need to enforce SSL certificates for Login either way ...

But people might easier have SSL then SFTP or FTP with SSL, so perhaps that is a step forward ...

Best Wishes,

Fabian

There are extensive issue

pwolanin's picture

There are extensive issue threads on d.o about the security and mechanism for doing in-place updates. They are off-topic here, so please discontinue.

agreed

Gábor Hojtsy's picture

Agreed that we can try and stick to this schedule. I'll talk to my employer about making time available when needed :) Drupal 6 will most definitely not have monthly releases, but we can aim for the last Wednesday of each month for sure as a possible release date.

Looks like this is settled...

webchick's picture

This proposal looks like it has buy-in from Dries, Gábor, and myself, along with a healthy chunk of major contributors. Let's dooooo eeeet!

Also, you guys totally scope creeped my nice discussion out of existence. :( SNIFF!

So please take further commentary to either:

Thanks! :)

Changes made...

webchick's picture

The Drupal core project page already linked off to http://drupal.org/documentation/version-info which contained a section on "When's the next release?" I simply added the bit of text above to that:

http://drupal.org/node/27362/revisions/view/1302666/1509072

Yay! :D

Small change: "if not, you're

star-szr's picture

Small change: "if not, you're safe until next the month" = "if not, you're safe until the next month"