This is part one of a four-part post. The rest of the proposal can be found at the following links:
- Part 1: Today's Project Application Process
- Part 2: Where the Existing Process Breaks
- Part 3: The Proposal
- Part 4: Sanity Check - Wait a Minute Here ...
The 'Coles Notes' summary of the actual proposal can be viewed HERE.
Or, if you prefer, download a copy of the entire proposal in Word format.**
Part Three: The Proposal
So … where do we go from here?
The rest of this document outlines a tiered ‘application’ process, with different ‘classes’ of project, and escalating levels of review.
In addition to improving on the issues described earlier, this proposal attempts to address the following process goals:
- Enforce Licensing requirements
- Enforce Security requirements
- Discourage project “Name Squatting”
- Encourage proper use of Drupal APIs
- Encourage consistency in the application of coding standards
- Discourage community from accessing/use of unsafe contributions
- Encourage community sharing of code/modules
- Discourage further proliferation of Module Duplication
- Discourage perpetual ‘beta’ cycles
Overview
The general approach with this proposal is to break the current project application process into a number of much smaller, more manageable steps; while still attempting to maintain the integrity of projects which are promoted to the equivalent of ‘full project’ status. Progression through each of the steps would be required to raise the project from one ‘class’ of project to the next; and each project class would come with progressively stricter review requirements.
The hope is that through breaking the journey from sandbox concept to full project into a number of smaller progressive steps, where the requirements for each of these steps are clearly defined, the process outlined in this proposal will:
- Allow new contrib authors to immediately get their project out the door without playing a ‘waiting game’
- Allow more immediate opportunities for feedback for new contrib authors, by simplifying the basic requirements at early stages of the process, thus facilitating more participation from the general Drupal community,
- Provide more opportunities for progression, and thus more opportunities for positive reinforcement (and the associated sense of gratification) which comes along with the clearing of each milestone,
- Provide a well-defined roadmap for new contributors, so that it is always clear what the ‘next step’ requirements are,
- Encourage continuous improvement of contributed projects as they work through the process, and
- Provide a framework in which participants can put in as much or as little effort as they want (and reap the associated rewards!)
Whether you phrase this as a ‘badges’ approach, a ‘ladder’ approach, a ‘gates’ approach, or a ‘staged progression’, the concept is the same … there would exist a few different classes of contrib projects, and progression between each class and the next would have its own set of tasks, checks, and requirements.
The 'Contrib Project' Classes Defined
As mentioned, the root of this proposal is centered around the use of progressive classes of contrib projects. These classes would roughly align with the regular ‘-dev’, ‘-alpha/beta’, ‘-rc’, and ‘-1.0’ release cycles … and progression between each class and the next would have its own set of project requirements.
This proposal doesn’t suggest how the different project classes would be labelled or displayed on Drupal.org … this is seen as a follow-up discussion which can occur if the underlying fundamentals of this proposal are accepted as an appropriate evolution of the full project application process.
"Un-Certified" Projects
Description
The first class of project would be the ‘Un-certified’ project. Un-certified projects are those which have no more than a ‘-dev’ release, do not have an official namespace, and would roughly equate to today’s ‘sandbox’ projects. In fact, un-certified projects would only be allowed to exist within a sandbox repository.
However, this proposal would open up limited release/packaging capabilities to these sandbox projects … allowing the publishing of –dev releases (and only –dev releases) within the sandbox environment.
Opening up the sandboxes to official –dev packages addresses the git issue outlined in “Pain Point 3: Sandboxes” earlier in this document; and it also lends greater legitimacy to sandbox projects (helping them feel more like ‘real’ projects), thus reducing the ‘sandbox stigma’ described earlier.
‘Uncertified’ projects would likely contain some type of ‘Not Verified’ disclaimer text on their project page, with a friendly ‘use at your own risk’ message. (For example, “Note: This project has not been reviewed to ensure security, compliance with Drupal coding standards, or proper use of various Drupal APIs”.)
Requirements
As the ‘entry-level’ class of contrib project, there would be no minimum requirements for the creation of an ‘un-certified’ project. Again, this is essentially an extension of today’s “sandbox” projects category … there is no security review, code review, or peer review required - anyone would be able to create and publish their –dev release within their sandbox repository.
"Pending" Projects
Description
A “Pending” project is one that has moved out of the ‘–dev’ state, and has started down the path towards an official point release. These projects would be granted an official namespace and non-sandbox project page (similar to a ‘promoted’ project today). Ideally, a ‘pending’ project would have at least a ‘-rc’ release; though the number of existing projects in ‘perpetual beta’ would necessitate either the grandfathering of any existing ‘full projects’ to ‘pending’ status, or introduction of another class to represent these ‘grandfathered un-certified’ projects.
Once promoted to ‘pending’ status, contrib authors would be allowed to publish any releases from their git repository. However, similar to Un-certified projects, a ‘Pending’ project would likely contain some sort of disclaimer text on the project page, and any ‘point’ release would not be officially included in the security team’s scope of responsibility until promoted to a ‘certified’ project. (A thorough review, roughly equivalent to today’s ‘project application’ reviews, would be one of the requirements before a project could be promoted out of ‘pending’ status.)
Requirements
Pending projects would need to meet the following requirements:
- Have an –rc release (possibly also allow alpha/beta – see sidebar discussion below)
- Meet Drupal licensing requirements
- Have a Formal ‘project page’ description
- Module runs through Coder cleanly (ignoring false positives)
- Meets basic coding standards (regarding namespacing, documentation blocks, etc.)
- Contain a list of related/similar modules (either on the project description page or in the issue queue), and explanation of how this particular project differs from each
Projects which did not meet all of the above requirements would not be allowed to progress to ‘Pending’ status, and existing ‘Pending’ status projects which were found to be in violation of these requirements (and did not address them in an appropriate timeframe) would risk being unpublished or demoted.
Sidebar: Motivation and Issues with the '-rc' restriction
The goal in setting a ‘-rc’ requirement for the ‘pending’ state is to discourage folks simply renaming –dev branches to –beta in order to claim a namespace, without actually making any changes to their code ... in other words, as a means to limit the potential for ‘namespace squatting’. In addition, it provides an additional incentive for contributors to upgrade their modules out of perpetual beta, and into one of the ‘more stable’ release categories, which should in turn help with adoption of their module and the overall Drupal contrib project status metrics. (Consider http://webchick.net/node/89)
The problem arises with the large number of existing projects in the contrib space, which only have existing –dev, -alpha, or –beta releases. Establishing a min release requirement for new projects before they can claim a namespace, when there already exists namespaced projects which do not meet the minimum requirements, will still lead to the appearance of a ‘double-standard’ for new project applicants.
This can be partially mitigated through how the classes are ‘displayed’ on Drupal.org. For example, if using badges, the grandfathered projects without an ‘-rc’ release could display the same ‘un-certified’ badge as a sandbox ‘-dev’ project. Alternatively, a new project category may be required to represent projects in-between the ‘un-certified’ and ‘pending’ states.
Triggers initiating the 'Un-Certified' to 'Pending' Progression
In addition to complying with the above requirements, a module would have to satisfy ONE the following triggers before being considered for progression:
- (Default) Exceed a minimum required ‘soak period’ as a –dev module.
- Essentially, modules would be promoted from sandbox to full project after a period of x weeks/months. This soak period allows time for the project maintainer to demonstrate competence as a maintainer through responsiveness, user interaction, evolution of the code, and active queue management.
- Having a time-based trigger also means that modules which are not widely used or publicized can eventually claim their own namespace as well … even if they are less well-known or have only one or two actual end-users.
- (Optional) Request an ‘Official’ peer code review
- In order to accelerate the progression from ‘un-official’ to ‘pending’ (ie. Bypass the ‘soak’ period), a project maintainer could apply via a project promotion issue queue, requesting a ‘peer code review’.
- The process for this review would be similar to the existing project application reviews, where another community member would validate that the project meets the minimum requirements for promotion, and then mark the issue RTBC to signal that the project is ready for promotion out of ‘un-certified’ status.
- However, the actual content of the review would be considerably less than the existing review process … and essentially consist of running the module through Coder, validating licensing requirements, and checking that functions and variables are properly documented and namespaced.
- In theory, anyone in the community should be capable of validating these checks (and quickly), rather than depending on a dedicated review team.
- If a review request is made, and for some reason it does not get addressed, the module would still be eligible for automatic promotion once the default soak period has expired (assuming that it meets the other base requirements as well).
- (Optional) Reach a certain threshold of reported Drupal sites using the module (Suggested: 50? 100?)
- If a module is widely used (as identified through the ‘Reported installs’ metric on the project page), it would be eligible for promotion without having to wait for the end of the ‘soak’ period or a peer review.
- Essentially, this provides a fast-track option for popular modules.
- In theory, higher usage should either indicate a certain level of stability, or at the very least result in higher levels of end-user testing, helping improve the stability of the module faster than one with a smaller end-user audience.
- (Optional) The ‘Community Fastrack’ approach
- This option would allow for community members with the appropriate rights to immediately promote modules to ‘pending’ status, without needing to meet one of the ‘soak’, ‘review’, or ‘threshold’ triggers above.
- This approach allows for the quick promotion and publicizing of important or long-awaited features/modules and other community-driven initiatives.
- This permission would be held by a restricted group of trusted community members, similar to the ‘Git Administrator’ role today.
- While restricting this permission could lead to the impression of favortism or classism if it were the only way to promote a module, having it as one of many avenues for promotion should help mediate this concern.
"Certified" Projects
Description
Certified projects represent the final step in the project evolution. Certified projects have at minimum one point release … and by the time a project is promoted to ‘certified’ status, should also abide by all security, licensing, and coding standard requirements.
“Certified” projects would become the primary focus of the security team (whereas “pending” projects with point releases would be a lower priority, until they are promoted to “certified”).
Requirements
To be promoted to ‘certified’ status, a project must exceed all of the existing requirements for ‘Pending’ status, plus:
- Have an official point release (ie. -1.0)
- Abide by all recommended Drupal coding standards (though many existing modules would likely be grandfathered)
- Undergo a technical code review for security, translations, and proper use of Drupal APIs (essentially the same as today’s ‘code review’ process).
In addition, the project maintainer should be able to demonstrate competence as a maintainer, through a history of active issue queue maintenance.
Triggers initiating the 'Pending' to 'Certified' Progression
In addition to complying with the above requirements, a module would have to satisfy ONE the following triggers before being considered for progression:
- (Default) Request an ‘official’ peer code review
- Project maintainers who would like to promote their project from ‘pending’ to ‘certified’ would request an official peer code review, essentially leveraging the same process that is used for ‘full project’ applications today.
- Once the review has been completed, and any issues addressed, another community member would validate that the project meets the requirements and mark the promotion issue RTBC to signal that the project is ready for promotion to ‘Certified’ status.
- Unlike the promotion to the ‘pending’ state, the ‘certified’ review would have a much stricter set of requirements, essentially mirroring those required for today’s ‘full project’ status.
- If a review request is made, and for some reason does not get addressed in a timely matter, the project maintainer could also pursue one of the other triggers which would make them eligible for ‘certified’ status.
- (Optional) Reach a certain threshold of reported sites using the module AND exceed a minimum required ‘soak period’ with at least an ‘-rc’ release
- This trigger is based on the argument that high usage infers some level of community acceptance of the module, and speaks to its stability/security (while not actually guaranteeing it).
- An associated ‘soak period’ is included with this trigger to provide sufficient time for bugs and other issues to bubble up to the surface via the project’s issue queue.
- The threshold for this stage would be significantly higher than that for promotion to the ‘pending’ state. (Suggestion: 300? 500? 1000?)
- (Optional) The ‘Community Fastrack’ approach
- Essentially, this option would allow for community members with the appropriate rights to immediately promote modules to ‘certified’ status, even in cases where the maintainer has not formally requested a peer code review (or in cases where the project does not have a formal maintainer).
- This permission would be held by a restricted group of trusted community members, similar to the ‘Git Administrator’ role today.
- However, community members promoting a module through this approach would still be responsible for creating a ticket in the project promotion queue, indicating they have reviewed the module and found no issues, validating that it meets the criteria for ‘certified’ status, and marking the issue RTBC.
- To complete the promotion, another community member (other than the one who created the RTBC ticket) would validate the ticket and review before promoting the project to ‘certified’.
- Alternatively, the ticket creator could solicit another community member to perform a review of their own and post a comment in the project promotion issue to confirm that they have also reviewed the module and feel it is ready for promotion, after which the original community member could promote the project themself.
This is part one of a four-part post. The rest of the proposal can be found at the following links:
- Part 1: Today's Project Application Process
- Part 2: Where the Existing Process Breaks
- Part 3: The Proposal
- Part 4: Sanity Check - Wait a Minute Here ...
Or, if you prefer, download a copy of the entire proposal in Word format.