IMPORTANT: Source code, projects, and commit access in a post-Git world

There's a 3rd option, that I think is the best of both worlds:

There would be 2 tiers of users (roles) with Git access.

A) Once you upload an ssh key and fill out the trivial stuff in your profile to agree to only upload GPL code, you have restricted project access. You can create sandbox projects, but you cannot promote them to full projects. To get a sandbox promoted, you go through an approval process as webchick describes (file an issue in the appropriate queue requesting a review of your sandbox by project admins who have power to promote other user's sandboxes to full projects).

B) Once you've had N sandbox projects promoted to full projects, and you've demonstrated you know WTF you're doing, you can apply to have your account added to another role that allows you to promote your own sandboxes to full projects (or just create full projects in the first place).

I don't think everyone currently with a CVS account would qualify for role B. Yes, we'd need to clarify and document the criteria for being promoted to the "can make full projects" role, so that's more work at some point. And, the project administrators would also have to review these kinds of applications, which is yet more work. But, it does allow our best and brightest to work unimpeded, might actually reduce the review load required if every project has to be approved (since no one would ever have to even look at a request from merlinofchaos et al), and still doesn't carry over the "once you're inside the front gate, you can wreck in the whole castle" problems of our current approval process.

On a technical side, the code for supporting sandbox projects in Project* was written to support any of these options, so it's strictly a configuration and policy question. No new code would need to be written. We'd just have to setup an additional user role on d.o. Effectively auth users would have 'create sandbox projects' perm, while this new role (name TBD) would have the "create full projects" perm. Users that have both perms see a checkbox when creating a new project to control if it's sandbox or full, and on any sandbox project they can edit, they can promote the sandbox to full.

So not a direct response yet, but that means that every project, even sandbox ones, will have a full issue queue with all of the related functionality? They just won't have release nodes?

Also, when a project gets "promoted" (however that happens) does that also then mean that its Git URL will change, thus breaking all existing clones of it? (I'm OK with that being the case; I just want to verify that I understand the implication.)

Let's not make this any more painful then it has to be, eh?

I'm wholeheartedly behind option #1. I really don't see how "approvals theoretically will go faster" because the sheer man(or woman)power behind having to review every. single. project.

It's not very often that messy and non-compliant code also works well. Code that doesn't work well is usually unpopular. Unpopular modules die off and are replaced by modules that do things right.

If implementing #2, I would have the review be as automated as possible. For example, you'd have to pass a coder.module audit to be granted approval, or have an approved reason for why you can't do what coder asks.

My $.02,

Justin

I'm going to stay within the lines and stick with 1 and 2.

Option 1 works, improves our current system dramatically, and only has a bit of our current problem at the end (the application process is painful both to go through and to moderate)

Option 2 is horrible, would impact our community in a horrible way. Never never.

Edit: I am way in favor of quality metrics on projects, using all kinds of axes.

+1 for dww's #3.

I really don't get the point of #2. If d.o doesn't want my code, then fine, github.com is more than welcome to take my code.

Option #2 wouldn't be as big of a problem for me if the review was more about following the coding styles and security reviews and left out the politics of whether a module is needed or not. We need to make sure that drupal.org remains a place (or becomes more of a place) where people can easily contribute their ideas and not be yelled down that their idea is bad. There are already many flamewars going on about forking modules, overlapping functionality and such that would become much worse with option #2 and would probably lead many developers over to github.

As such I'd vote #1 until and if we find a better way.

Keep things as they are. Yes, we have some cruft, but by adding sandbox projects we might have much less cruft. Incremental is the key! We can always add Option 3 later if we don't like things.

"You cannot create releases of sandbox projects. The only way for users to get the code of a sandbox project is with git clone."

I am hoping this does NOT mean that you are removing the 'tag' component of git? I would still like to be able to tag specific points in the codebase. Even if you don't want to make a release from that tag.

Oh, and I would vote for the modified #3 (or #1 if #3 is dis-allowed).

It's currently too hard to get in, but once you're in, there is no oversight and it becomes a hotbed of bad (and good) ideas.

As mentioned earlier in this thread, if oversight is too stringent, the ecosystem will starve. And nowadays, Drupal.org isn't the only option to host code. There are a ton of code communities, with Github being the most notable one. Anything that makes it harder to contribute will have continually increasingly negative effect on d.o. contributions as social coding sites continue to gain popularity.

I'm a bit intrigued by #2, but let me throw out a big spin on it for discussion's sake: Let anyone and everyone create a project but have the community maintain well curated lists of projects. Perhaps there can be a community tag system for module searching and ordering. "This project release passes coder," "This project was last audited by a security team member on xx/xx/xx", "This project is in use on over 1,000 sites" and maybe last but not least, "This project has received overwhelming community approval." Something like a gold star.

That way anyone can contribute, the barrier to entry becomes nearly nil, but module usage and community eyeballs will cause the wheat to separate from the chaff.

Duplicate modules, bad ideas and sub-standard projects wouldn't be highlighted and a variety of factors, not just the judgment of a couple of the right people will cause the project to be tagged in a way that it gets shown in default searches.

These tags, or badges would be welcome and useful identifiers to newcomers and evaluators. if you stick to the community curated modules, you would be significantly less likely to run into bad projects and duplicate projects than you would now.

I would have much more interest in reviewing applicants/projects/code if I could see them once they've gotten used how to commit code to drupal.org, use coding style, and have a place where I can review them easily (and not having to download random .zip files), which is why I'm for project-level reviews. And even a long-time contributor can forget to search and accidentally create a duplicate module. I'd be ok with my own new projects undergoing a review.

I vote option #1. The git migration process should happen unabated by this separate topic. I like the idea of #2 and a lot of the ideas discussed above are worth exploring too, so lets do that separately and allow more time for everyone in the community to work on a consensus.

I think many of the problems of project proliferation are being solved merely by the power and flexibility of both improvements to Drupal core and a variety of key modules.

And besides, once you are in a git world the culture of easy cloning to me seems to argue against any excess approval process. There will be a tipping point where you've made it just a bit too painful and off folks go to maintain their projects on github or wherever which I don't think the community wants.

Also, kudos on the sandbox project idea!

Option 1 is a clear winner to me. We've come this far (Drupal 7) and it just works. Are there occasional issues. Sure there are. But these are handled in the proper way as well - through the existing issue queues. We don't need to fix what's not broken.

The other suggestions get into reviews that are destined to be more political and don't, at the end of many long debates, really solve the problem. Each of us can certainly point to situations where established maintainers, who would certainly have full access in #3 post make full projects that others question for any number of reasons. The beauty of the community is we have forums for handling these questions and having the discussion. At the end of the day it works out -- the wonder of a do-ocracy.

Also I hope we could come up with a way to search sandboxes. Not offering this level of collaboration, whether on a problem, a bit of a theme or something that might become a full module leads to either duplication or somebody doing the indexing/cataloging off of d.o and ultimately diluting the value of our community hub/resource we've spent so much time improving.

I think a couple valuable points were brought up in support of dww's #3, where new and trusted contributers have separate permissions.

tim.plunkett mentioned that co-maintainers are given full access to create modules without the same review process as a brand new contributor: it is assumed that the maintainer that supported them did a sufficient review but there is little guarantee (e.g. a new contributer grants an even newer contributor access because they need the help and it was offered).
Several people have also mentioned that many contributors are sufficiently trusted by the community to create quality projects. I think it would be frustrating for a new contributor to view another project approval application that has been approved with little comment just because of who the author is, while their application is sitting in the queue unreviewed or with thorough criticism; they may not understand the history and reputation of the other contributor and feel slighted.

As Crell said though, the criteria for promotion (and demotion) must be clearly defined. Another concern is that the process for subsequent applications isn't an additional deterrent for contributions; it should be possible to say "you've already passed the hard part, we just want to check over things quickly" so that they don't decide another application is just too much work.

For subsequent applications, reviewers can focus on the areas that were issues for the initial application, hopefully speeding up the process. A valuable test is then available for granting someone un-reviewed module creation privileges: have they created a new project without repeating their original mistakes? Have they sufficiently supported their previous module, and continued to improve upon any issues brought up in their original review? If their subsequent applications go through with little comment, it is likely that their future efforts will be of equal quality.
This may not be at the same point for all reviewers. While it may be a good idea to require at least 2 successful applications to achieve trusted status, some contributors could take much longer if their first efforts are consistently incomplete (and then be encouraged to develop ideas in the sandbox further before applying).

First: Yay sandbox. This means I can put my github drupal stuff here, where it makes sense for it to be, so that everyone can pick it apart and put it back together as they need.

Second: Projects should be on a code basis, not a user basis. That is, there should be no access setting for you to automatically promote your own projects beyond sandbox. The project should stand by itself. It might work like this:

1) You add at least one co-maintainer to your sandbox project. All contrib should have co-maintainers before they're contrib.

2) Add an issue of a new type, something like 'request contrib status,' where you plead your case in public. Only one of these issues is allowed per project.

3) In the subsequent thread, anyone can change it to 'deny contrib status' and prevent you from requesting it again for a month (or some reasonable amount of time). I could say: "Your module doesn't have a .test file.... denied."

4) In the thread, any non-maintainer can give you an 'approve contrib status.' If you get some number of these without any denies for a month, you're in the queue for an abbreviated approval process, making it much easier on the set of people tasked with that queue.

5) The final approval/denial must also be given in public on the issue thread, and if they deny the project, the process can be re-started with another 'request contrib status.'

This is a peer review by actual peers, in public. Everyone learns coding standards, security concerns, and dev tools just from reading the process thread.

This takes a lot of pressure off those with the power to approve or deny the project, because it's clear from the thread if there are problems or not. This reduces the possibility of denials seeming to be political or somehow wrongheaded.

Someone who's a Drupal personality would still have to go through this. Their reputation might make it easier to find all the people involved to vote it the thumbs-up, and might intimidate deny trolls away, but that's not a problem because reputation should count for something.

So this is my vote for a modified Option #2, which would no doubt involve a great deal of coding. From a practical standpoint, it's probably better to go with Option #1 for the time being and revisit this after the git infrastructure is in place.

I would be for option 2 if you could create releases and search. What is the point of restricting those options? I would just make it very clear that it is a sandbox and not search them by default.

There is enough crappy modules or unmaintained modules already in the namespace.

If you would allow searching and releases, then all we are talking about is protecting the namespace and helping newbies out by cleaning up the junk in the sanctioned projects.

For the git migration launch, I fully support maintaining the status quo, ie. #1

If the plan of providing some new features on the project pages (reviews, five star, etc.) comes to fruition, then I would support maintaining #1. (It is nice as far as lowering barriers to entry, and with the sandboxes, I would support being more lenient on initial reviews.)

That said, I have real concerns about the continued cruftiness and duplication of projects in contrib. So if there wasn't a good rallying to get/keep contrib cleaned up, then I would fall back to #2, where each project needs to be reviewed before an official release. (Yes, I understand all the arguments against this, but I believe it could be a supportive, collaborative process with some fairly major changes and more people reviewing, which obviously has always been a bottleneck...so hopefully 1 will pan out with the added features.)

I generally dislike #3 because I think creating arbitrary tiers of contributors might be ridden with fail.

The sandbox idea really gets me pumped. I'm working on my first module over at github and I can bring it home.

(and, by proxy, #3)

[Note: This is an extremely opinionated rant by webchick, the community member. I'll post later as webchick, the Git migration team member, and it will be an objective summary of the discussion at hand for those who can't keep up with 80+ comments.]

Too much change, too suddenly

Contributors are already going to be dealing with three major shifts as a result of this move:

1. Re-learning how to maintain a project all over again (copy/pasting new commands, learning that 'git checkout' means an entirely different thing than 'cvs checkout', getting themselves out of totally new scary error messages when things go wrong...)

2. Coming to terms with the idea that anyone at all with a drupal.org account who checks a box can now contribute code, and how that will ultimately impact the community and contribution workflow.

3. Coming to grips with the contribution landscape now that there are "sandbox" projects, and how those will ultimately impact the community and contribution workflow.

So piling yet another change that has profound community and cultural implications seems like an enormous risk to me.

We can't keep up with the volume

New CVS application roll in at a rate of about 2-5 per day. The oldest ones have been sitting there waiting for approval for over 11 weeks. In general, people only approve applications when they feel they have some expertise over what the module is about or if they know the person personally. Reviewing is hard work.

New modules, on the other hand, come in at a far more rapid pace. There are 11 so far today, 5 yesterday... and this is a holiday week. We could work up a backlog of module approvals much grater than the number in the CVS application backlog within about 3 weeks.

So who are these mythical people who are going to review and approve these modules as fast as they come in, and why aren't they currently helping with our CVS application backlog right now, where we so desperately need them?

We need to remove barriers to contribution, not introduce more of them

Contributors are the lifeblood of our project. Without them, we have no Drupal. We have therefore bent over backwards to make it as easy as having a drupal.org account + some basic tools where necessary (diff, patch) to contribute to almost all aspects of the project: core, documentation, translations, participating local user group, issue queue triage, patch reviews, patch contribution, test authoring, etc. This is a good and happy thing.

To contribute new code, on the other hand, you currently need to jump through these hoops:

• not just a semi-working, but "release ready" module/theme
• scouring of all drupal.org projects to make sure module/theme doesn't exist yet (and hope it didn't spring into existence while you were making said release-ready module/theme)
• an in-depth understanding of coding standards
• an in-depth understanding of Drupal APIs
• an in-depth understanding of how to write secure code
• application form submission
• tar or zip
• an abundance patience to wait for an incredibly busy volunteer to respond (which, btw, might never actually happen)
• incredibly thick skin to take the comments of whoever reviews your application non-personally.
• more abundance patience to wait for a volunteer to respond again so you can repeat the process again a few times.
• irc (to raise the fact that no one responded to your application once you'd finally resolved all the issues)
  and finally, if you're very, very lucky...
• cvs

This makes absolutely no sense to me. People write crappy documentation, contribute incorrect translations, and submit half-baked patches. But that's great, and we welcome it. Why? Because it gets peoples' foot in the door of our wonderful community, and because even the most "needs work" attempt at something can be corrected, and improved upon by others.

So ideally, I'd want contributing new code to be the same as everything else on d.o which is: "Just Do It!" However, the security team has threatened to kill me. ;) So therefore, I opt for the next best thing, which is only force people to jump through hoops once, and then never again. (Option #1)

In the age of github.com and where 0.01% of Drupal.org users contribute to the project, we can't afford to throw unnecessary barriers in front of contributors. If we do, we will lose Drupal.org as our central collaboration hub, which has been our project's single biggest strength for its past decade of existence.

Duplicate projects, crufty projects, badly coded projects, etc. are legitimate concerns but should not be tied to granting contributors access

Why? Because there are many other ways of dealing with those issues:

• Reviews/ratings on project nodes (already slated for a future phase of d.o redesign)
• "Badges" on project nodes to denote test coverage, coder module compliance, etc. as a quality indicator
• A feed of new sandbox projects, which can be reviewed by the http://groups.drupal.org/similar-module-review folks who can chime in as things come in before a bunch of code is written and the project is ready for approval.
• "Opt-in" code reviews to request feedback from the http://groups.drupal.org/code-review folks.

In other words, we should be adding metrics to Drupal.org in order to help the good projects rise to the top, not dragging down enthusiastic contributors with rules.

I'm sure I have more to say on this topic, but that's enough for now.

Aside from the obvious inability to effectively review new modules by an ever changing group of volunteers, a review wall would simply add to the growing hierarchy within the community.

Here's a summary of the 80+ comments so far:

Weighing the options

There were several concerns raised with keeping the status quo (Option #1):

• The contributions repository is filled with crufty, duplicate projects and poor-quality, difficult to understand code that lacks documentation.
• The Drupal project has reached a scale and momentum where there almost always IS a module for that, and one-shot projects and duplicate efforts no longer have a place.
• Sandbox projects opens the door for forking, which means more duplication and less collaboration. Not placing controls around this means we could end up with chaos.
• Since oversight is only done at the beginning, contributors still create duplicate projects later on, which leads to end user frustration/confusion and wasted effort.
• The ability to create themes does not equate to the ability to create modules and vice-versa. Right now, it's "all or nothing" access which is incorrect.
• Even long-time contributors can make mistakes in their modules, and would benefit from someone looking over their shoulder before publishing a project to the wider community.
• The effect #2 would have on an average contributor is overblown. Most people do not create more than a couple of new projects a year.
• There are several ways to "hack" the existing approval process, e.g. become a co-maintainer. It's also a process that people only need to go through once, after which they're in the "in" crowd. This creates an air of unfairness that would be addressed by forcing all contributors to be in the same boat.

However, there were also several concerns raised with introducing more oversight (Option #2):

• Competition is seen as a valuable thing which allows innovations to occur like Admin module alongside Admin Menu, Ubercart alongside E-Commerce, etc. which likely would have been denied had their respective authors needed to go through an approval process at the time.
• The decision of what modules to use should be in control of the community and our users, not Drupal.org admins. Good modules will naturally float to the top, crappy modules will naturally sink to the bottom.
• This option prompted statements like "will break the community" and "our long history of having a rich and diverse module/theme repository would be put in jeopardy." It's a major cultural shift, and we're safer to go about change in an incremental way.
• The existing approval process is already keeping out major contributors and tons of code, even though it's a one-time thing, because of the politics people feel they have to endure. Introducing an even more stringent approval processes will drive even more people to host their code off-site or not share it at all.
• Git brings with it a culture of easy cloning, and fighting against this is a losing battle.
• It's not in scope for the Git migration (a technology issue) to impact policy (a community issue).
• There are serious reservations about reviewers being able to keep up with the volume of new projects coming in.
• The complaints people have with the existing "free-for-all" process can largely be mitigated with better searching and filtering tools on Drupal.org. Several options were suggested here, including community tagging, security team audit indicators, community approval ratings, badges for test coverage + coding standards, user reviews, a ModuleRank™ overall auto-generated rating, etc.

Sandboxes

• The idea of "sandbox" projects looks like it has broad support. A couple of people mentioned that they can now move their github code back "home" to Drupal.org, which is awesome and exactly what was intended. Yay! :D

• On the flip-side, it was pointed out that promoting sandboxes to a "full" Drupal.org project status introduces the possibility for user confusion as to what the "real" Views module is as opposed to someone's fork where they're playing with something. It also likely increases the chances of intentionally forked projects because of disagreements with maintainers or impatience. This breaks "collaboration over competition", which is a fundamental community tenant, and currently basically a requirement because there is only one canonical Views module, for example, and github.com/foo/myviews doesn't appear anywhere on Drupal.org.

• On the flip-flip side ;) while forking is easier to do with Git, so is merging in forked changes.

• There were some concerns regarding the findability of sandboxes which could use more discussion.

Counter-Proposals

A few counter-proposals to the two mentioned in the original post have been suggested:

dww proposes a middle-ground Option #3, where all code contributors default to having each new "real" project approved (Option #2), with an option to apply for being exempt from the checks for long-term contributors who are known to post consistently good stuff. This has pretty wide support, as well as some detractors who worry about the "two-tiered" contributor system it creates and how to keep that fair.

catch proposes moving the approval process from "real" project creation to instead approve the creation of the first RC or point release of a module (at which point it becomes the security team's problem). Let's call this option #4. This would result in more security oversight than currently exidsts, without impacting every contributor and project.

mdekkers proposes an Option #0, which removes any and all restrictions around committing code and welcomes module duplication, abandoned modules, etc.

Dustin Currie proposes option #1, but modified with the addition of a community-maintained curated list of awesome projects. Something akin to "Golden Contrib" which is an idea that's been kicked around for a number of years. Let's call this option #5.

"Votes"

In terms of raw vote numbers (which is a terrible basis upon which to make a decision, but good for spotting "trends" nonetheless…), here's how these shake out currently:

Option #0: 1
Option #1: 29
Option #2: 5
Option #3: 10
Option #4: 1
Option #5: 2

(here's how I counted those)

Option #1 on the surface appears to have a ton of votes, but it's worth pointing out that some of those want the ability to re-visit the decision post-launch once we see how things shake out a bit. Option #2 appears to lose miserably, but when coupled with #3 (which is really just a modified #2) the numbers are fairly strong, though still only about half.

Conclusion

Given that there's not overwhelming consensus to go with #2/#3, and some serious concerns expressed by folks about introducing more approval/oversight, #1 still appears the safest way for us to go for initial launch.

However, this discussion is very good because it's bringing to light a bunch of frustrations and ideas that we can try and flesh out more for the future. So please, let's continue. :)

I do not want to be a second-class citizen, under any circumstances. I do not want a super-special, secretive committee deciding whether I'm a big boy using an arbitrary process that depends on the weather on Mars that day and what they had for breakfast. I also do not want to have to be effectively "Drupal-certified" to be a first-class citizen. And I do not want yet another system that is painful for the majority of Drupal users, but which the "inner circle" doesn't care about because it doesn't affect them.

I don't care if we create a process to review every single new project (or its first release) or not, but if we do it, everyone needs to be subject to it. I don't think I should have to wait until someone gets around to reviewing my issue in order to create a new project, but that's "just" a logistical issue, assuming all such issues are treated equally.

I'm no fan of duplicate projects, particularly forks, but I'd much rather take action against them retroactively. A simple "flag as possible duplicate" button or something like that would suffice. Then the offending projects could be reviewed and moved (back) to sandbox status if necessary after giving the creator a chance to explain. Option #6, perhaps, although that's really more or less the way we do it now.

We often talk about how the Drupal community is great because anyone can contribute freely. Come for the software, stay for the community, right? We are one community, not a "trusted" community and an "untrusted" community. So a big thumbs down to #3, unless the promotion to the "trusted" role is automatic and based on a very low, very transparent threshold such as ownership of a single project that passes Coder and has been in existence for 3 months.

As a lighter alternative to #4 how about a review system for releases which is open to all contributors. Something like this:
I release 7--1 of my new project so I email a couple of other contributors I regularly work with and ask them to review the release. Once they have done so and are happy they tick a checkbox and the release is shown as "Reviewed by 2 people" on the module page, with a link to a page showing how many reviewers, who they are and comments. The security team audit can remain as it is - not required for a release - so that good code doesn't get delayed by an overworked team.

This would allow long-time contributors to utilise their standing in the community to bring in heavy weight reviewers, whilst providing a good way for high quality new contributions to quickly bubble up.

We could have a "Modules I reviewed" page as part of their profile, so if I want to find out what Views related modules merlinofchaos has reviewed I can find them easily.

I am getting Webchicks point that the workload to review every new proposed module would be beyond reason. And also it is against the open spirit of Drupal.

To the outside world something else is important: how to find the best module for my task? We need to provide a lot more metrics/ratings/sorting criteria/comparison criteria on module listings and pages. It has gotten enormously better with the new d.o. and can still go a long way.

It can be 50.000 Modules as long as the most popular and best maintained are easy to find and it will harm no one.

+1 for option #1. That will surely encourage more contributors, hence, more modules and improved ideas which will definitely help in the growth of drupal powered projects

Improving the existing working option (#1) with sandboxes is evolutionary, it enables Drupal contribution to grow and gives tools for better evaluation by those doing the approvals. In short, it's fantastic, cause it doesn't muck with a working ecology, even if the speed at which approvals happen is slow there's nothing to say that those approving applications cannot grow with the community. #1 is fertilizer.

Option #2 creates opportunity for politics and preference to undermine a meritocracy. Even if it creates an upside for code quality/security, that quality would be present in #1 anyways, as it is now. As Webchick stated above where would Ubercart be if it and E-Commerce had come out at the same time under #2...and from there where would Drupal Commerce be? Non-existant.

Largely I say #1 so that we leave module selection to the user, the uninitiated user innovates accidentally and outside the box, and will find novel and unconventional uses for Drupal as the framework grows more, and more flexible. Ratings systems, comparisons and even the fact that competing modules often join forces down the line will help with the issues created by a growing and largely unregulated community.

Cheers :)

I will keep my support for option #1 or even option #0. I'm generally in favour of duplicate modules and letting them live or die through natural selection. However, there will always be room for enhancement. A few that I can think of:

• automating the CVS/Git account approval process as much as possible.
• automating Coder reports for modules.
• don't require Coder compliance, but require maintainers to annotate any issues from Coder before they release.
• encourage built in unit tests.

Maybe the best way to encourage quality is a point system based on: a) Coder compliance, b) security reviews, c) performance reviews, d) automated test coverage, e) user votes, f) issue queue stats, etc.

I think most of the above have been suggested, although I'm not sure I've seen automated test coverage, or performance reviews. I'd hesitate to include usage stats since there will be some niche modules with great quality that never have a really large installed base.

I think we need to separate this into two discussions:

1. Continue this discussion on what is required for the initial Git migration, any project workflow adjustments that really can't be lived without otherwise major problems would be created. Clearly this would be option #1 with possibly some minor adjustments (see above for some ideas).
2. Start a separate discussion on how to expand upon the initial release to further improve everyone's lives, both to not overburden the existing community with additional manual review processes but also to add indicators to help guide visitors towards more stable code & away from unstable code.

Keep in mind too, a MODULE cannot be vetted by security review in perpetuity, only a RELEASE. In the future it might be nice if the security team could place a stamp/seal on a particular release. A "at this point in time (7.x-1.2) this module was secure." It would give the developer one more tool to maintain security.

After further review, I'm still in favor of #1:)
The Module quality assurance process should be divorced from the sandbox vs full project issue.

I think we need to take a step back and consider what our goal here is. There are a lot of conflicting goals in play, and that colors everyone's perspective.

1) We want more warm bodies. More developers == good. This is taken rather axiomatically. However, just because this statement is part of the Drupal Faith doesn't make it true. Is it? Is simply increasing the numeric quantity of people who write code that lives on drupal.org a good thing? If your answer is "Well of course, duh", then you are doing Drupal a disservice.

2) We want "good" modules, where good is some combination of "code that is fast, bug-free, security hole-free, and conforms to coding standards". Well, sure. But why? Because Drupal the product is not core. Drupal the product includes all code available from Drupal.org. Security "firms" (sometimes legit, sometimes trolls) routinely refer to the number of SAs released against "Drupal", and include all contrib SAs. When we talk about the strengths of Drupal, anyone who knows anything quickly points out that core is useless without contrib. Extensibility is what makes Drupal worthwhile, and more importantly what it's extended with. Drupal 6 without Views and CCK is, to be perfectly honest, a complete waste of time as a CMS. The "official" contrib repository is part and parcel of Drupal the product, and it is on Drupal the product that the developer and professional reputations of all of us is based.

3) Because "Drupal the product" includes Drupal.org, we want the quality of the repository in toto to be high. Quantity of modules != quality of the repository. Having multiple modules that do the same thing and differ only because someone didn't apply a patch fast enough for someone else harms the overall quality of Drupal the product, because it reduces collaboration and increases confusion for users looking for modules.

4) Because "Drupal the product" includes Drupal.org, but not GitHub, we want the "good" modules to be on Drupal.org, not GitHub. My company produces multiple modules for most sites we build that are never released, mostly because they're not useful to anyone due to being too specific. More generic modules, though, are useful to more people and we try to release those. The corrolary to that is that yes, not all modules are equally good. Some are crap, some are useful to exactly one person, etc. Others are useful to nearly every site. The latter are the ones we want improving Drupal the Product. This is also why we have an SA process for all (released) modules on Drupal.org; they are part of Drupal the Product.

5) We want "good" developers, because "good" developers product "good" modules. Some built-in review step is, really, our only opportunity for more experienced developers to help newer developers become newer "good" developers rather than newer crappy developers who produce crappy modules.

6) Drupal the Product, because it includes Drupal.org, includes the people on Drupal.org. That is, the Drupal Community is part of Drupal the Product. This ties back to goal #1, but the caveat is that not all developers are equal. "Good" developers are more valuable than crappy developers (for some definition of good and crappy). Some crappy developers can be turned into good developers with more or less guidance, but not all. (Of course, there is no way to point that out without violating the western-Anglo-Enlightenment axiom that everyone is equal in potential, and to say otherwise is creating an "inner circle/elite/cool kids" class and "plebs". That statement is of course utter BS, but we cling to it anyway.)

7) It is also true that, since crappy developers can in many cases be turned into good developers, we want to keep them around long enough for them to turn into good developers rather than scaring them off to GitHub where they will either not improve Drupal the product or not become good developers, or both.

There are also technical constraints to consider, especially the fact that we have a single namespace. There is one, and only one, "filefield" module in all of Drupal the Product. It lives at http://drupal.org/project/filefield. That is the only and only filefield module. Anyone who doesn't think that http://drupal.org/project/ldap and http://drupal.org/project/ldap_integration aren't confusing is diluding himself. The former is the "better" namepace, and what someone is likely to look for first. But how many people know the history of those two namespaces? Which really is the "real" module? A "just throw anything anywhere" approach will lay waste to the namespace, a problem we are already seeing. Hence the proposed "no rules" #0 approach is not even worth considering. It is logistically not even possible.

Drupal.org is not GitHub, nor does it aim to be. Drupal the product benefits from not having projects forking on a daily basis. We do not want to encourage that, and if someone does routinely fork modules those forks should be considered second-class citizens. Yes, there, I said it. Note that module duplication is not inherently the same as module forks. Those are two subtly different issues.

So given all that, the underlying question here is "what structure will best serve Drupal the Product?" Our goals are: More developers. Higher-quality developers. Higher-quality modules.

Let's try to keep the conversation around those goals, and an understanding of them.

The whole issue of using Git is being seriously over-engineered. Git is a decentralized revision control system, we've all heard that, but it's such a hugely important point. It's why Git was written in the first place. If you haven't seen Linus' talk on Git, go see it at YouTube: http://www.youtube.com/watch?v=4XpnKHJAok8 . Even if you have seen it, see it again, and pay attention to what he says about network of trust, commit access, patches and tarballs, branching and merging. In particular: "So in practice... there will be a thousand or maybe twenty thousand different branches, but in practice you won't ever see them, because you won't care. You will see a few main branches, maybe you will see only one--In the case of the kernel, they really only look at my branch, so even though there are lots of branches, you can ignore them. The way merging is done is the way real security is done, by a network of trust."

There is no canonical repository with Git. With Git, Linus is no longer "The" Linux kernel (short of legally owning the trademark, but with respect to the actual code). This fact is being entirely lost by people who still want to think Drupal.org hosting is keeping the same centralized model with commit access. But it doesn't exist. It is easy for anyone to commit code now (without resorting to keeping a backlog of patches), and the only discussion that is relevant is "What subset of code will Drupal.org voluntarily host, out of all the code being stored in Git repositories?" Drupal.org is not the authoritative source for what modules may be published, it is just one place out of many to get hosting (it might be the only place to get Drupal.org branding, etc, of course). There's no reason that every single project hosted here couldn't up and move to Sourceforge.net or GitHub.com right now and continue development in a free environment with no significant regulations except that it's posted under an open-source compatible license. A more likely solution is that projects will start at GitHub (since it's cheap to start a repository there), then go through the (less painful nowadays) process of creating a Sourceforge.net project, and then once it meets all the criteria for Drupal.org hosting, it comes over here. Is that what we want? Or is Drupal.org a place where anyone can publish their code, publish their branches, and easily collaborate?

Having thousands of branches (each repository has its own branch, remember, unlike CVS where there the branches only exist on the server) is the reason why Git is so popular, and makes collaboration easier. Git is not an excuse to play by yourself in your own sandbox, it's a reason to collaborate easier with other people. That's even the slogan of GitHub.com: "Social Coding". If the site is looking to be successful, it needs to at least emulate those features that made it successful: branching, merging, social coding. That is: As few regulations as possible for hosting code, since commits are no longer scarce economic resources, and possibly higher standards for being assigned a module name which is in limited supply. Note how GitHub gets around this, by identifying repositories with the username... it's doubtful Drupal.org could use this because it's assigning more than the project name, it's assigning a globally unique module name. And there's the answer:

Don't focus on Git, Git is the wrong problem. Focus on where there are significant economic resources: Naming modules. That's what Drupal.org does best, not hosting code, if we want to determine who gets a module name, focus on that problem instead. Otherwise there is no excuse for restricting code if the intention is to create a vibrant community. (If Drupal really wants to solve the economic problems associated with naming, it should move to using URIs for naming modules, but that's a discussion for another target audience.)

kyle_matthews wrote:

In any case, there always has been duplicate/redundant/bad/stupid/insecure/whatever code and there always will be. So if people have trouble finding good modules I vote that we develop better filters not stop projects that someone doesn't like from being created.

I want to highlight this, because it gives real insight into a currently existing problem that will not be solved by either of the two initially proposed options and indicates a solution.

How about not discouraging duplicate / similar modules but providing improved methods for a) developers & site maintainers (infrastructure group I guess?) to tag related modules, b) visitors to see related modules to better find the one that suits their needs. Maybe something as simple as either an extra vocabulary or even just expanding the Module Categories list to add hierarchies would be sufficient?

A key part, though, would be finding a good way of displaying this information on the project page itself - maybe replace the current "Related Modules" list with a) a list of "Complimentary modules" (e.g. XMLSitemap and Nodewords are complimentary) b) a list of "Similar modules" (e.g. Nodewords, Metatags and Integrated Metatags are similar/duplicates)?

But this could be functionality for after the initial migration.

Just saying.

I see android market vs. Apple market. Sure there's less "crap" in the Apple app store then the Android store (also debatable statement I know) but the Android store "feels" more inclusive and open. I see less people complain about the process that google has in-place with android then the overlord approach of Apple. Yes, you'll get bumper crop, modules that "don't do things the right way", poorly coded, but you'll also have more eyes / voices on the big projects because people feel more included in the community.

I'd also hate to see winners and losers picked by an approval process. Yes there would probably be more high-quality modules (and less of them) but I think the usage statistics, committer track record, and maintenance of the project page often speak volumes. What about a prominent "What to look for in a maintained module" page / video on the module search page. Give some rules-of-thumb, show what they look like and let usage statistics drive winners and losers

I believe that #1 or #2 doesn't really matter as the sandbox system itself will be the game-changer.

Background:

I'm one of the last people with 6-digit user IDs. :P

In ~3 months with Drupal, I've written three modules (pretty tiny though – some 10kB in total), and another one is in the works. These modules are used on a production site and pass Coder review. I've not bothered to apply for CVS account, mostly because my modules are specific enhancements to the core poll which (I reckon) would be of interest probably to two or three sites in the whole world.

When sandboxes are introduced, I'm going to put most of my stuff out there; I'll consider any comments or suggestions that might (or might not) come in, but I won't care about projects being 'official' or not. Perhaps I'd even prefer to keep them in the sandbox.

Somehow I suspect I'm not alone in being only happy to share but not giving a damn about status.

Lots of issues here, but the main one circles around the idea that there are 'too many' modules. Even duplication is only really a problem when it is combined with the sheer volume of modules.

Volume is a problem for developers because it's hard to even figure out which module might solve your problem before you create (yet another) solution. Volume is a problem for users because it's hard to separate the wheat from the chaff.

But I would argue that diversity is good. Even if there are three modules to solve a problem, the first one may or may not solve it well (or well enough for your use). And it's not always possible, or even desirable to create gigantic modules that incorporate every single feature that any other developer wanted to add. Some maintainers are open to incorporating your extra code, but in many cases it's difficult or impossible to do so without creating some Frankenstein monster that is hard to maintain and really doesn't make anyone happy. It can be done and it has been done, but it's not automatically 'better' to pour everything related to one topic into a single module. And from the user perspective, I often want to find the simplest module that solves my problem, not necessarily the one with every possible bell and whistle.

The assumption is that the way to reduce volume is to put up as many obstacles as possible to keep modules from being created. And that creates another problem -- finding people with the interest and energy to review the applications and determine what is 'good' and what is not. In reality, you won't review a module until you need it, and you review it by trying it out and finding out its strengths and weaknesses. And most modules aren't well-used until they have releases, not before that. Once released we have a huge base of people who might find problems in them -- not just the security team.

So how could we solve both problems? By keeping the freedom to create modules and projects but doing a better job of filtering them and cleaning out the ones that aren't working or aren't well maintained. Some of this is already implemented -- we now have a list of 'most downloaded' modules and maintenance and download stats on the project pages. We could use a richer taxonomy for filtering them, and a better job by maintainers of tagging them appropriately.

And maybe we need a better system for 'demoting' modules. If we added some easy ways for the general public to report that a module is a duplicate of a better-written module or that it is poorly written or poorly maintained, or, obviously, that it has security issues, then the security/maintenance team would only need to follow up on those reports rather than review every project. Maybe in our new system we can 'demote' projects back to sandboxes -- no code is lost, they are still there, but less visible.

I am for option #1.

Any option where all users who have a CVS account would not be able to create full projects is unrealistic.
What would happen if a project is necessary for drupal.org, or any sub-domain? If it is necessary to have N sandboxes project to create a full project, then nobody would be able to create a project for Drupal.org, where a project is sometimes created only to have a project issue (the Drupal.org webmasters, Drupal.org CVS applications, Drupal.org infrastructure, Git Migration Community Testing, Drupal.org Project applications, Drupal.org Redesign, Test driven development infrastructure, and Documentation projects).

Sorry, too late. We are going with using sandbox projects.