IMPORTANT: Source code, projects, and commit access in a post-Git world

webchick's picture

(TL;DR? Here's summary #1:

For those who have not been following along with the CVS -> Git migration progress, this post contains important information for you to be aware of.

Things that we know:

  • Once Git launches, anyone with a account will be able to commit code to
  • In order to get Git commit access, you will need to read and agree to the Developer Terms of Service in your user profile which includes items like "I will only upload GPL compatible code" (basically, the same agreement people have to agree to on the CVS application form right now).
  • There will from now on be two types of projects you can create: "sandbox" projects (which all users can create) and "regular" projects (which approved users can create). They share basic common features--a project page, committer access, issue queues, etc.--but with a few key differences:
    • Sandbox projects get a numeric shortname repository based on their project node IDs (like 12345.git) to indicate that they are "experimental" so that they don't take up a "proper" repository's namespace.
    • You cannot create releases of sandbox projects. The only way for users to get the code of a sandbox project is with git  clone.
    • Sandbox projects are not visible in search results, listing pages, main issue queue listings, etc. Basically, they can only be accessed from the author's user profile.
    • There will be some visual indicator on a sandbox project node page to indicate that it's experimental (Exact nature TBD: See
    • The security team has no responsibility over code in sandbox projects.
  • A sandbox project can be made into a regular project at any time via a checkbox by those who have permissions.
  • There will be an approval process of some kind, similar to the existing CVS account approval process, but far less daunting and time consuming, because:
    • It will focus only on "big impact" items: overall understanding of Drupal's APIs, security review, license compliance, blatant duplication of other projects.
    • Reviewers can use standard tools like commit logs, diff, and the issue queue to understand the code and its evolution and get a "sense" of the maintainer, thus removing a lot of pain and guesswork from the approval process.
    • All non-critical follow-ups (coding standards compliance, naming conventions, etc.) can be handled in the standard issue queue way, and addressed by the author either prior to or after their sandbox makes the transition to a regular project.

The last piece is deciding how the permissions to create "real" projects are allocated.

Option #1: The "Keep things as they are" approach:

  • All pre-existing CVS account holders will receive permissions to promote their own sandbox projects to real projects; their access wouldn't change.
  • If you are not a pre-existing CVS account holder (which means all new module/theme contributors post-Git migration), you will need to go through a one-time approval process in order to get these permissions.
  • Quality control standards on new projects aren't changed in any way.

Option #2: The "Approval for every project" approach:

  • All projects must go through an approval process, regardless of who the author is.
  • Approvals theoretically will go faster because everyone's in the same boat, from merlinofchaos to someone with a 7-digit user ID. (though it's possible that special roles could be granted to prolific contributors)
  • A higher standard of quality control will be placed around new contributed modules, because each one must pass a review of their peers.

Because the migration team's goal is to minimize the impact of the CVS to Git migration, both for community health and time/budget reasons, and because option #2 could always be implemented at a later date if wide consensus is reached that it's a good idea, the Git team strongly advocates implementing option #1. In other words, we keep the status quo for existing and new contributors, and do not introduce additional approval mechanisms into the contribution process.

We are opening a period of community feedback until no later than January 5, 2011. Please speak now or forever hold your peace. Or at least hold it until after the Git migration goes live (currently scheduled for mid-February 2011). :)


Option #3

dww's picture

There's a 3rd option, that I think is the best of both worlds:

There would be 2 tiers of users (roles) with Git access.

A) Once you upload an ssh key and fill out the trivial stuff in your profile to agree to only upload GPL code, you have restricted project access. You can create sandbox projects, but you cannot promote them to full projects. To get a sandbox promoted, you go through an approval process as webchick describes (file an issue in the appropriate queue requesting a review of your sandbox by project admins who have power to promote other user's sandboxes to full projects).

B) Once you've had N sandbox projects promoted to full projects, and you've demonstrated you know WTF you're doing, you can apply to have your account added to another role that allows you to promote your own sandboxes to full projects (or just create full projects in the first place).

I don't think everyone currently with a CVS account would qualify for role B. Yes, we'd need to clarify and document the criteria for being promoted to the "can make full projects" role, so that's more work at some point. And, the project administrators would also have to review these kinds of applications, which is yet more work. But, it does allow our best and brightest to work unimpeded, might actually reduce the review load required if every project has to be approved (since no one would ever have to even look at a request from merlinofchaos et al), and still doesn't carry over the "once you're inside the front gate, you can wreck in the whole castle" problems of our current approval process.

On a technical side, the code for supporting sandbox projects in Project* was written to support any of these options, so it's strictly a configuration and policy question. No new code would need to be written. We'd just have to setup an additional user role on d.o. Effectively auth users would have 'create sandbox projects' perm, while this new role (name TBD) would have the "create full projects" perm. Users that have both perms see a checkbox when creating a new project to control if it's sandbox or full, and on any sandbox project they can edit, they can promote the sandbox to full.

+1 on option 3

jhodgdon's picture

I like this option, and it doesn't sound like it is really too much more work than option #1. Letting just anyone create any project once they've gotten one approved (the way things are now) has led to all kinds of cruft in the projects.

And a big +1 for the git migration team, by the way, and the idea of sandbox projects in general -- sounds like a really good idea, and it will remove a big barrier of people sitting for months in the issue queue waiting to have their CVS license application accepted, since anyone will be able to sandbox at any time.

This sounds like a good

cweagans's picture

This sounds like a good option. If no new code needs to be written, I think we should go with this method.

There's a lot of code in CVS that doesn't follow coding standards or doesn't work or is difficult to understand because it's not documented. If we went with this method, then we could more or less ensure that at least 80% of newly published projects would be quality code with good documentation.

The only thing missing here is the conditions to get the "I'm a badass" role where you can elevate your own projects to full projects.

Cameron Eagans

+1 with concerns

Crell's picture

I like this middle ground, with one caveat: It inherently creates a two-class environment for module developers. Now, we should not knee-jerk reject that concept. It is legitimately worth considering. However, that also means we must consider how one graduates from "junior module developer" (needs one of the older kids to bless every module) to "senior module developer" (can stay up past 10 pm). Has X number of approved modules? X number of approved modules that don't suck, for some arbitrary definition of suck? Hasn't had a security bug reported in X weeks? Does that mean if someone reports a Views security issue that Earl gets demoted? :-)

I am open to this line of discussion and think we should have it. That will determine whether or not we can adopt #3, which I do believe is the better approach if we can come up with a fair way of deciding who gets the self-promoting role and who doesn't.

+1 for option 1

mike stewart's picture

1 all the way. the only truly fair way to control the code chaos is to just let the users decide - best code wins. creation of barriers and people will go away.

mike stewart { twitter: @MediaDoneRight | IRC nick: mike stewart }

Risks with #1

Aveu's picture

While hopefully it does not happen, "best code wins" could become gamed by politics and personalities.
I trust the senior developers here not to play that way but having spent a few years participating in Wikipedia RFC's I am loath to see that approach create schisms here.


Crell's picture

So not a direct response yet, but that means that every project, even sandbox ones, will have a full issue queue with all of the related functionality? They just won't have release nodes?

Also, when a project gets "promoted" (however that happens) does that also then mean that its Git URL will change, thus breaking all existing clones of it? (I'm OK with that being the case; I just want to verify that I understand the implication.)

Actually, it wouldn't "break"

cweagans's picture

Actually, it wouldn't "break" anything. The url would likely change, but all you'd have to do is either a) reclone the repository, or b) do git remote rm origin && git remote add origin

Cameron Eagans

When projects get promoted,

sdboyer's picture

When projects get promoted, they'll become accessible at both the nid-based URL and the shortname-based URL. Both will work, and under the hood the shortname repo will just be a symlink to the nid repo.

The proposed features for

mikey_p's picture

The proposed features for sandboxed projects are (note these will be configurable at any time):

1) use nid as short name, instead of custom full name - This is not expected to impact clones as our git server can abstract the repo name you're asking for from the location on disk. Worst case, it'll be symlinked or something like that.

2) Won't appear in global project listings, i.e. not indexed by solr

3) Have releases disabled

4) Visually distinguished from regular projects

5) (still up for discussion) Won't appear in global issue queue listings

Option #1

sirkitree's picture

Let's not make this any more painful then it has to be, eh?

Yes I agree that option #1 is

gordon's picture

Yes I agree that option #1 is the best method in the short-medium term as it is not really any different to the current process is now, and should not cause any huge influx of projects that are going to get quickly forgotten about.

And as it says above we can always build #2 later if we feel that it will give us an overall improvement in the quality of projects.

Gordon Heydon

Option #2 is already performed somewhat by the community

justintime's picture

I'm wholeheartedly behind option #1. I really don't see how "approvals theoretically will go faster" because the sheer man(or woman)power behind having to review every. single. project.

It's not very often that messy and non-compliant code also works well. Code that doesn't work well is usually unpopular. Unpopular modules die off and are replaced by modules that do things right.

If implementing #2, I would have the review be as automated as possible. For example, you'd have to pass a coder.module audit to be granted approval, or have an approved reason for why you can't do what coder asks.

My $.02,


#2! "Is there a module that?"

sun's picture

Is there a module for that? -- Yes. In 99% of all possible cases.

The Drupal project has reached a scale and momentum, in which it almost always no longer makes sense to think small (a.k.a. one-shot projects), or to duplicate efforts (waste everyone's time).

Especially if everyone's able to rethink, revamp, and redo existing things at any time and without restrictions in sandboxes, it's becoming increasingly important to restrict the various possibilities of...

  • oh, you have objections to my major rewrite? well, ok, I've forked it into a new...
  • I spent the last 3 weeks with rethinking this whole approach and created a new project, which works fantastically for me... sorry for letting you know so late...
  • I've used that module in the past, but the maintainer wasn't very responsive, so after more and more stuff got changed, I created a new...
  • This is the same as ... just with a better UI, because the other one sucks.

If we provide better tools, then we need to take the consequences into account.

I can only get behind #1 if there's a formal time-line/schedule for doing #2 (prior to 2012).

Daniel F. Kudwien
unleashed mind


geerlingguy's picture ;-)

I vote for #1, or #3 (if that's an option). But I really would like to get behind #2 (just not until a later date... Drupal 7 release + Git migration completion should be first goals here).

Option #1

jcfiala's picture

My gut feeling is to lean towards #1, so that's what I'm voting.


Option 1++ Option 2 breaks the community

rfay's picture

I'm going to stay within the lines and stick with 1 and 2.

Option 1 works, improves our current system dramatically, and only has a bit of our current problem at the end (the application process is painful both to go through and to moderate)

Option 2 is horrible, would impact our community in a horrible way. Never never.

Edit: I am way in favor of quality metrics on projects, using all kinds of axes.

If it ain't broke...

mrf's picture

From the perspective of a long time user and more recent contributor I'd say the current approval system works.

It took me a long time to gain CVS access and now that I am more advanced I can fully understand the reason for denying my initial request for access. This single barrier seems to be effective in weeding out potentially troublesome contributors, and the new system would provide much more information about the person being reviewed and their abilities while still allowing them to get their code out there.

Are there enough examples of abuse and misuse of project creation access to put more barriers in place for people that have already jumped through all the necessary hoops?

+1 for dww's #3.

Dave Reid's picture

+1 for dww's #3.

Senior Drupal Developer for Lullabot | | @davereid

A Vote of NO on Option #2

mbutcher's picture

Please, please, PLEASE do not go with 'Option #2: The "Approval for every project" approach'. This is giving way too much authority over what gets hosted and what doesn't. Drupal's long history of having a rich and diverse module/theme repository would be put in jeopardy. The result would likely be that people avoid the politics and go to GitHub (which is compelling when both D.O and GitHub are git repositories).

Not addressing mbutcher's

sdboyer's picture

Not addressing mbutcher's whole argument, but just this specific bit:

and go to GitHub (which is compelling when both D.O and GitHub are git repositories)

I don't think really needs to be so scary any more. If anything, it's a benefit. If folks really want to see/use some of the github features, that's fine. It'll be trivially easy to get it back onto d.o, and fact is that unless/until it does, releases can't happen. We'll see how it all plays out in the long run, and maybe I'm just deluding myself, but I just don't think this'll seem like such a threat anymore. Especially when we roll out per-issue repos.

Option #3 (dww's suggestion)

tim.plunkett's picture

Subscribe to the new modules feed for a couple weeks, or spend 5 minutes looking through the Duplication Hall of Shame, and it becomes clear why Option #3 is needed.

EDITED: After re-reading dww's suggestion, #3 > #2

I really don't get the point

mikey_p's picture

I really don't get the point of #2. If d.o doesn't want my code, then fine, is more than welcome to take my code.


sun's picture

As explained above.

Most recent and actual real-world examples I know of are

If there are good reasons to duplicate efforts, then there are little arguments against chaos. But in almost all cases, duplicated efforts and projects are effectively introducing confusion (on all ends), harming the Drupal project and product at glance.

Projects are products.

Daniel F. Kudwien
unleashed mind

I expect that we'll see less

cleaver's picture

I expect that we'll see less of this as soon as people can create a sandbox project. The fact we're switching to Git should make it easier authors of forks to work with maintainers.

Then we need to stop and

mikey_p's picture

Then we need to stop and define what's goals are. The whole point of this migration and sandbox projects is to let more people post projects and their code to and get wider exposure. If that's not the case, and's goal is only to host a curated list of approved modules, then fine, I'll just take my code to github, which doesn't care.

I guess in a round about way you are agreeing that if people want to find a place to stick some code to get some exposure, they'd be better of going to github than d.o?

a place to stick some code

sun's picture

a place to stick some code

You're not talking about real projects/products then. For that, we already have and will have sandboxes.

I'm actively using my sandbox already, not sure whether you are. I can stick my arbitrary ideas into that, and people can use that code and perhaps even improve it or discuss with me. Heck, they can even fork it, if they want to.

But that's just code. This topic is not about code.

The CVS applications queue is full of applications that are duplicating existing efforts, and, as someone has mentioned elsewhere in this thread, even prolific contributors create new projects that are duplicating existing community efforts, just because they didn't know they exist.

By providing git, we're providing a much easier tool for forking entire code-bases into new code-bases. And since sandboxes will be free for all (contrary to CVS sandboxes right now), we're providing an essential tool to fork code in an easy way and keep it under version control - heck, even having an issue queue and project page attached to it!

This means that the average John Doe will be able to fork Views or whatnot into a sandbox, spend weeks or months with changing and improving code/plugins, without ever contributing back any piece of that, and lastly, that user will have sufficient arguments to tell us: "Hey, I've rewritten major parts of Views for my needs; it's worth an own project now."

"Look, I even resolved 1,947 bug reports that have been filed against my sandbox!"

We need to foster collaboration and contribution to the Drupal project at glance in a solid and reliable way.

Daniel F. Kudwien
unleashed mind

There always have been forks

kyle_mathews's picture

There always have been forks and there always will be forks on and elsewhere. Yes, Git makes it easier to fork but it also makes it much easier to merge back divergent changes. If anything, moving to Git will reduce the number of (permanent) forks because instead of being forced to create a new project to maintain their changes, they can maintain the changes as a branch, maintained in parallel, until a point where they and the maintainer(s) can agree on how to merge their visions.

Anyways, forked projects aren't that big of deal (what's a few extra megabytes) and sometimes they turn out to be a good thing (e.g. Ubercart). Sometimes people who fork projects are just annoying gits (pun intended) but sometimes they actually have a better idea how to solve a particular problem and the divergence between their vision and the original maintainer(s) is too great to bridge. Yes they're annoying but generally one module will emerge as the best solution to a problem and other competing modules will cease being used (and maintained). At each version upgrade many modules go through this filtering process.

And to make a broader point, we're debating whether a input-filter strategy or a output-filter strategy is best. Should we filter projects before they're created (through some sort of review process) or filter projects after they're created (by community chatter in the forums and elsewhere (hey what's the best module for x), popularity rankings (how many sites use x), and other metrics). I vote for the latter :)

In any case, there always has been duplicate/redundant/bad/stupid/insecure/whatever code and there always will be. So if people have trouble finding good modules I vote that we develop better filters not stop projects that someone doesn't like from being created.

Kyle Mathews

Here here!!

bleen's picture

In any case, there always has been duplicate/redundant/bad/stupid/insecure/whatever code and there always will be. So if people have trouble finding good modules I vote that we develop better filters not stop projects that someone doesn't like from being created.

I couldn't agree more.

Preach it.

sdboyer's picture

Preach it.

Confusion, or conflicts of interest?

mdekkers's picture

Daniel, it comes as no surprise to me at all that you are crying and hand-wringing about projects that implement things similar to modules you have developed or maintained. Why don't you compete, instead of complain?

All this talk about "waste everybodies time" and "harm the project" and "chaos" and all that other fearmongering, combined with the "regular" Drupal politics, is what has whittled down my companies enthusiasm for contributing code over time. We are not a "famous" Drupal shop and mostly keep to ourselves, but we are extremely prolific, have a rather sizeable, dedicated Drupal development team, and produce tons of code. We currently have, in house, about 150 distinct modules that we use and re-use. Some are unusable outside our firm, simply because they deal with hyper-specific client functionalities, but a good selection are generic, and likely very usable for other community members.

For example, we have a pretty awesome set of integration modules for Drupal and SugarCRM that allows you to manage paid site subscriptions to different site sections from within sugar, integrate various aspects of sugar into the site, such as helpdesk tickets for example, and lots of other similar functionality. We have a totally kickass newsletter module that completely blows anything publicly available right out of the water, and then some (it's younger sister is a Drupal project - managed newsletters, but we didn't keep it up to date. I requested CVS access for my Lead Developer, several times, and this was either ignored, or rejected on stupid technicalities - we gave up with the whole Drupal projects scene soon after that).

Long before the current, popular SOLR implementation was born, we had a Drupal / SOLR implementation (which we still run and maintain) that has had, from more or less the beginning, multi-core instantiation from within Drupal, high availability and failover, the ability to use SOLR as a content cache, per anyting (node, taxonomy, whatever) index options and settings, hot, cold and single node index rebuild, and again, a ton of other stuff. I am not current with what the "Drupal" SOLR project is up to right now, but at the time most of this was not yet available. when we looked up from implementing our SOLR stuff, the Drupal SOLR project was already in place, with some of the drupal stalwarts on the team. There was no way that we would put out a different project - not because I thought that to somehow be the wrong thing to do (competition is awesome) but because of such ridiculous stuff like the duplication hall of shame (should be called the "developer insecurity and-don't-play-on-my-patch-land hall of shame") and the inevitable politics that would ensue. We actually gave the SOLR project our code some time ago - maybe there would be something of use there for them - but they didn't have time to look at it. They did, however, strongly agree that launching another SOLR project would be not right. I firmly believe that having our SOLR code public would have been a net benefit (and still would be) for Drupal as a whole - not because one or the other project is better but because they are different. I just dont have the time or energy to deal with people like you, so I am keeping my ball to myself.

Like some other fine commenter mentioned a bit earlier "If Drupal doesn't want my code...." - we have chosen to support Drupal in a different way (cash, mostly - we sponsored, directly and indirectly, a few cons and the association)

I see, from the same old crowd - of which a large proportion are the same tired old whingers that keep banging on about module duplication, a very strong direction towards creating a multi-tier participation structure: the "in" crowd, and the "plebs" - this is one thing that has always disturbed me somewhat about the Drupal community, and is something that has always informed our decision to not engage heavily in community participation - we are too busy for the drama, and life is too short to deal with the stupid Drupal duplication police. The whole hall of shame effort is not even useful, and floats on the broken and deeply flawed idea that duplication is bad. You know what - different people have different needs, and like different things - maybe I just want to put a simple, stupid google map on my "contact us" page, without pulling in a complete geolocation framework into my code that can have intelligent and autonomous conversations with the Google API. Can I have that, please, without somebody yelling about having his e-peen stepped on? All this, to me, is just posturing, and appears to be mostly driven by a need to be seen as better, more "in" and somehow, more part of the Drupal community.

Open source is about duplication. It is the absolute, utter, complete, no-bullshit framework for duplication. Duplication is competition, and competition is AWESOME. it drives innovation, new thinking, and sharing of ideas, code, and points of view. So what if some sap decided to fork some other module, and then proceeds to only make minor mods to the css file? who cares? Good projects will float to the top, and not so good ones will drop to the bottom.

Here is some constructive criticism for those brave enough to have plowed all the way through my rant: What we need is less crying from people with vested interests and positions to protect, and more tools to help inform and drive decisions for end consumers about what modules to use. The current "Insecure Developer Hall of Shame" should be framed and held up to posterity as a good example of misguided energy, and swiftly be replaced with a framework where module developers can add their own module, and list specific functionalities within a grid, and add some comments about why their implementation of the default submit buttons drop shadow is much better then the other 15. For people that are currently having issues with the default submit button, having choices and the ability to easily compare those choices is actually useful. A bunch of devs wringing their hands and crying in their beers about how somebody else dared to actually do something that was already done by somebody else - not useful to anybody, and in my view, does significantly more harm to the project then 50 different guestbook forks will ever do.

So, in conclusion of this fine, epic rant, I propose an "Option #0":

Let everybody, everywhere, create unlimited projects freely and without restraint;
Every user will have a tickbox in their profile "enable development tools" with a suitable "here be dragons" warning, and a small note about etiquette and whatnot;
Stop worrying about irrelevant stuff such as abandoned modules and whatnot - it matters to some and doesn't matter to others;
Stop trying to control everybody;
Put some energy into creating a module comparison framework, that complements (or replaces) the useless "here is a list of 12 million modules, in A-Z order";
Allow project maintainers to fill in random fields such as "provides api", "requires xyz" and "provides admin interface" or whatever, alongside functionality listings, so that I can compare stuff quickly and easily;
Allow for a free-form one liner to describe special features "My code is so optimised and efficient, it will actually ADD extra RAM to your server magically" - you get my drift.

I vote Option #0

Well that was quite a rant. ;)

webchick's picture

I actually don't disagree with most of the sentiment expressed here, though I do wish you'd express your opinions without resorting to name-calling and the like. It's bound to turn people off from the "meat" of your post, which is actually sound.

However, there's a practical problem with option #0 (which would otherwise be my vote as well) which is that the Drupal security team issues SAs for not just core but all projects in contrib. And that's a huge benefit to the project. So they've naturally thrown up severe concerns around the idea of a true "free for all", because under the current rules each contributor at least goes through some kind of an approval process before they can unleash their code on the world and the security team is responsible for it.

So the least invasive option to my mind that we can get away with is still #1, unfortunately. Catch's idea of moving the approval process to only the first "stable" release node is perhaps a middle ground; you could still give your code away unhindered in "real" projects, but limited to just -dev or -alpha/beta releases. Do you have thoughts on this?

Also, I should point out...

webchick's picture

That while duplication == competition == innovation is one philosophy, the "collaboration over competition" model that our community encourages is also what results in one Views and one CCK module that a gazillion people improve and extend, rather than 13 or 14 different "silo" modules that do slightly different things and have slightly different quirks, like some other projects have in these areas.

So I think it's a balance. We're generally much better off if there is one very well-supported game in town for a particularly difficult architectural problem space (CCK, Views, Features...), but the more user-facing the modules get, the more having a variety to choose from is an asset, not a detriment, as long as the differences are made clear to end users scratching their heads and trying to figure out which one to choose.

I do not disagree with this,

mdekkers's picture

I do not disagree with this, but I believe that ultimately the success of code like Views or CCK is not only because they filled a need, but because they did so in a robust, elegant, and architecturally sound manner. It is not a case of "we were here first" but more a case of "we did it -mostly- the best way". Out of a bunch of silo modules, somebody would eventually do it right, get contributions, grow, and thus become the major player. However, that does not invalidate the fact that there can be good reasons to do something similar, but different. I re-iterate my Google Maps point. Do I really need to pull in the whole gmap/locations or gmaps/cck combo's, if all I want to do is show a simple map? and how about the awesome work that development seed is doing around mapping? should that be frowned upon, because we already have mapping tools?

Webchick - please do not misunderstand me - I get the arguments, and I understand them. It is just that I am of the firm belief that approval processes for 3rd party, stand alone contributions to the project are a slippery slope, highly imperfect, and never likely to fill any kind of meaningful need - we should acknowledge that, and go on to build infrastructures that turn these issues into strengths, rather then systems that try to suppress them. I fully agree with your sentiment that "We're generally much better off if there is one very well-supported game in town for a particularly difficult architectural problem space" and actually believe that this extends not just to difficult problems, but also simpler needs. I believe, however, that this should be an organic, Darwinian process, as opposed to depending on some kind of politburo, supported by the duplicate-module hunters. Because at the end of the day, between that shameful hall of shame and some random approval process, we are not doing much at all to engender collaboration and participation.

While I think mdekker's long

coderintherye's picture

While I think mdekker's long post could have been more succint, he does have a point, which is that there are several abusive module maintainers who go out and attack new modules with no solid basis.

The worst of this was when a "Wall of Shame" was created by Rob Loach and he went around putting links to it on people's project issue queue. He didn't even do any due diligence, just a 'Oh, this looks like something I've seen before, wall of shame time.' Despicable.

I'd love to have one module do one thing the best, but not at the expense of killing off the community or turning off new developers so that they don't want to contribute back. There are enough self-promoters and dead modules for us to go after out there before we should even begin to worry about cutting off druplicate modules that actually work. I'm not stranger to going after new committers either, I lobbied hard to get commit access taken away from w3bsule and all associated accounts after it was found that they created project pages for 8 modules, but the only code they committed was a direct rip copy and paste from one of the SEO modules.

So, again, why go after legitimate developers, even if they are reinventing the wheel to some extent, when there are plenty of completely non-productive modules, abandoned modules, outright spammers, and empty project pages with no releases still exist on d.o.?

Drupal evangelist.

Security team only responsible for stable releases +1

rayvaughn's picture

This sounds like a sensible idea to me. We tend to regard dev and alpha releases with deep suspicion, and tread lightly with betas as a matter of course. I think it would be very useful if the first stable release of a project had some kind of review before being published. That said there is nothing to stop all manner of badness being committed after that...

Well, yes security reviews

mdekkers's picture

Well, yes security reviews are important, but I got my CVS access, and did manage to create quite a few project without ever talking to a security team person, as far as I can tell, so I am not sure how this would impact one way or the other. Moreover, I am not even a coder, myself, and just uploaded code on behalf of my company, so again, not sure how anything here supplies a comfort blanket to the security team.

I really do appreciate all the (hard and thankless) work that the secteam perform, and it is really important for me, and no doubt everybody else that works with this codebase. However, people are increasingly releasing to github as opposed to d.o and there will, no doubt, come a time when a project there will seriously take off and become ravingly popular. If issues are found, and reported to the secteam, about such a module will they turn around and say "sorry, not our problem"? That would raise serious questions about the purpose and work of the secteam, and their remit and scope. Alternatively, if they would state that of course they would deal with this, regardless of where the code was hosted, then we are in a situation where the "free for all" already exists - I am really curious about these "severe concerns" the secteam has, and would appreciate some link if those concerns are public (apologies if this is a well trodden path, I just jumped into this discussion on todays post).

I guess what I am saying is that I am not sure I can see the concern the secteam has, in the light of the fact that anybody can write code anyway. If it is actually about "well, but when you put it on d.o its different" then we already agree that we have a community of castes. A security team that only reviews a limited set of code - well, I guess that it has limited use.

I strongly believe that open source is about free participation for all that want to join in, and it would be totally awesome if could provide such a framework. A gated community is just no fun at all, and in my view, invites a particularly nasty brand of politics that unfortunately, we already see plenty of here. "Being Published" on d.o is in absolutely no way any kind of measure of quality, and I am sure that we can all agree on that. So once again, I don't understand / see how there is any kind of practical difference between sandboxes and full projects, except for the rather arbitrary limitations that are currently set to be implemented. I am having a really hard time to see this in any other way then "we want to reserve the ability to have a cool namespace for the cool kids" - sorry for the cynical attitude...

Finally, about approval processes in general - what are the criteria? What are the metrics for a "stable release" - who makes these rules, and why do they matter? Looking at the state of affairs today shows a good example - coding standards. Our company coding standards differ in several minor, but non-trivial ways: how tabs are implemented, for example, and a few other things. I really do not see any kind of way where I would ask our developers to change their code to be fully compliant with the drupal coding standards just to release our work. I do understand that for many companies, specifically very drupal centred shops, releasing code to Drupal is an important part of their overall marketing strategy, and therefore a worthy investment of time. We on the other hand do Drupal, and lots of other coding work - Drupal is an important part of our work, but not even close to the only thing we do, and that makes it a lot harder to have to change everything just so that we can give some stuff back - we really do want to give stuff back - lots of it, actually, it is just very hard to so so now, and with the upcoming changes I am afraid it will become even harder. sorry for the tangent - to come back to the main point: who would now get to decide that my code is stable? No, sorry - not a fan of approval processes. Let the issue queue speak for itself (that is a big part of how I personally decide what modules to implement, and I am sure I am not the only one. Is the maintainer responsive? are there many issues, how long are they open, etc. etc.)

Finally, I apologise for any perceived namecalling in the sense that I don't mean any offence - just a very strong disapproval of whatever I am calling names about (if that makes sense...)

However, people are

catch's picture

However, people are increasingly releasing to github as opposed to d.o and there will, no doubt, come a time when a project there will seriously take off and become ravingly popular. If issues are found, and reported to the secteam, about such a module will they turn around and say "sorry, not our problem"?

That's exactly what the security team does at the moment - for modules on github, on, and it's completely fine as a policy - if you don't use's release processes, issue queues (and hence bypass update.module and the rest), then you don't get security announcements for security issues either. Sometimes individual security team members will contact module authors anyway, but that's off their own bat.

However I will say that the current CVS application process currently doesn't do a lot for the security team - since the code that's reviewed during the cvs application process, isn't going to be the exactly the same code (sometimes not even the same module) that makes up the person's first release. Hence 'option #4' - if we're going to have a barrier at any point in the process, it might as well be just before this point, rather than before the code even gets on at all.

If people actually wanted to go for a no barriers at all approach (I get the impression webchick would quite like this, obviously you would), then we could also discuss with the security team changing the policy on what modules get supported.

Currently, any 'stable' release, even if it doesn't work, has five users, and is riddled with XSS holes, has to go through the entire SA process if someone reports a security issue - which means someone on the security team contacting the maintainer, dealing with back and forth patch reviews, co-ordinating the actual security release and SA, often having to review the rest of the module, not just the bug that was reported, for other security issues - all this process takes at a minimum more than an hour, sometimes several - for a very small and already overstretched team.

It would take another round of discussion, but the policy could be changed to support 'only modules with stable releases and more than than x users'. Or there are workflow changes being discussed to try to let maintainers co-ordinate and release their own SAs - but there has to be something in place to stop an even greater workload being made for the security team - either as part of that workflow change or to stem the tide before it's in place.

Every new stable release on is potentially about to sap several hours of someone's volunteer time who was not responsible for it being created in the first place. That issue has to be taken account in any decision that gets made - and for me personally this is one of the main reasons I'm currently extremely inactive on the sec team - because seeing people's time get wasted fixing issues they didn't create in modules that no-one uses (not every SA is like that, but enough are) makes me feel very bitter.

Despite being the creator of the 'Duplicated Module Hall of Shame' group (it's a shame you didn't appreciate the tongue-in-cheekness of the title, or the fact that many of the posts in there are exactly the comparison tables that you suggested people do instead of having that group), I don't really care about duplicate modules that much (although individual cases still annoy me sometimes - modules where the only difference is that the project name is singular or plural for example) - since that group was created, we've got project usage stats, d.o redesign, solr etc. all of which help to filter things out, and that is always going to be more successful than a manual review process. But I do care about anything which requires manual grunt work from volunteers, and that describes both the current cvs application process, and having no process at all - without very significant changes happening elsewhere.

Hey Catch, so - am I correct

mdekkers's picture

Hey Catch, so - am I correct in assuming that the security team doesnt look at any code until an issue is brought to their attention (I kind of always assumed that this was the case)?

Yes this is correct.

catch's picture

Yes this is correct. Obviously individual members of the security team end up reviewing modules for sites etc. and may find security issues, but that's not the role of the security team itself.

I just wanted to split this

mdekkers's picture

I just wanted to split this out from the other discussion... I think most people understand it was tongue in cheek (I did), but for various reasons, some people (like me, and you know from the original thread that I wasn't alone. I don't think I even bothered responding there) just took it the wrong way for various reasons (again, I did). I actually subscribe to the group, and check it out on a regular basis, and appreciate the constructive effort some people make in creating the comparison tables. Mikeytown for example, is very good at coming up with good lists and roundups. There is though, a group of people that are more of the witch-hunt type, and are just about "name and shame" and whatnot - that just disturbs me greatly.

also, oddly relevant, in a

mdekkers's picture

also, oddly relevant, in a tangential kind of way,y.0,no.,content.true,pag...


naught101's picture

I think I have to vote for option #0 now :D

@webchick: I think it'd be possible to think of criteria that define the limits of the security team's work without requiring any kind of prior approval. A simple one that comes to mind immediately is a threshold minimum number of sites using the module, say for example, only modules that are used by 1000+ sites in the last month are security reviewed. I think this would be far superior to the current system because it would remove the need for any rushed, possibly ill-informed and subjective approval process, while having the added benefit of removing old and abandoned modules from the to-review list as they drop in popularity.

Of course, if/when module reviews are implemented on d.o, that would provide another bunch of metrics that you could create limts from (eg. only modules that 50+ reviews with an average score of 60%+, and no "more security holes than a rusty seive" tag).

Basically, the meta-idea would be to somehow make the approval process community driven, and stochastic, and not a limited to a small pool of (possibly stressed, over worked, or even biased) volunteers who then have an even harder job of keeping up with the accounts they've approved to make sure that no-one has gone over to the dark side in the mean time.

Just to clarify, the security

catch's picture

Just to clarify, the security team doesn't actively review any modules, there is not nearly enough time for that. What it does is respond to security reports about modules - which often involves reviewing the module in question to find similar issues. The current policy is somewhat described at

Changing that to include "only modules with > 1000 users" is something that is definitely worth discussing - it would remove most concerns I personally have about dropping any kind of application process, but it likely needs a separate issue and explicitly inviting people on the sec team to comment - since that discussion has already happened (I remember it being done quite definitively 2-3 years ago, may also have been re-hashed since), and this would be re-opening it.

Well, that answered my

mdekkers's picture

Well, that answered my question in my previous post. So, if you don't actively review everything right now, then what would be the problem with letting everybody do their thing? How would that create more work? I am all for "you only get secteam time if you have [insert meaningful metric]" - that makes a lot of sense. I know we have more then 4 or 5k modules, but seriously, most of those can be sorted with a message to the maintainer, and a big red banner on the page "this module has a potential security issue - please contact the maintainer" or whatever, if something got reported.

Security team burden

webchick's picture

"So, if you don't actively review everything right now, then what would be the problem with letting everybody do their thing? How would that create more work?"

Well, it creates a lot more work if we don't also change the policy of what the security team is responsible for. If there's no approval process, I expect the number of projects and new people creating them to go up at least 10-fold. Without the initial "You need to remember to check_plain() your strings before your print them" and "don't forget to put placeholders in db_query()" check that (almost) all current CVS account holders have had to go through, it's a recipe for quintupling the security team's workload, which would be a complete disaster.

However, I would be 10,000x in favour of some metric like "installed on > 1000 sites" or whatever (and later possibly adjust that to take into account ratings, etc. but live installs is quite a bit harder to game) before it becomes the security team's problem. catch, where would be the best place to kick off that discussion?

Also, I do think that code reviews are valuable. But the current review process could be handled like this instead:

  1. A view that pulls in a feed of projects from people with (only one project) || (an account < 1 year old) || (some other arbitrary criteria) and have people volunteer at the group to monitor these and provide feedback to the authors.

  2. Some sort of opt-in process to say "I want a code review" which results in a badge on their project page "Last code reviewed on XX/YY/ZZZZ by XXX" or something. This would allow volunteers who enjoy providing in-depth code reviews to be paired up with people who want that level of scrutiny, and it would be a great indicator to end users that "Oh, this project has at least been sanity-checked by someone."

slight variant

yktdan's picture

webchick: Also, I do think that code reviews are valuable. But the current review process could be handled like this instead:

A slight variant on that is "I use it on my site and it has been flawless". Or "I tired to use it on my site and I had to spend all my time applying endless updates." Even non-coders can make a useful contribution.

The last discussion I

catch's picture

The last discussion I remember was on the security team list, I e-mailed that list pointing people here.

Do you have the URL for that

naught101's picture

Do you have the URL for that previous discussion?

That practical problem ...

Liam McDermott's picture

However, there's a practical problem with option #0 (which would otherwise be my vote as well) which is that the Drupal security team issues SAs for not just core but all projects in contrib. And that's a huge benefit to the project.

What is the huge benefit of the security team supporting all of contrib vs. a subset of modules that meet certain criteria? If it's that users shouldn't be directed to a module that's not supported by the security team, that can be fixed by excluding modules which don't meet those criteria from d.o. search results and listings. A warning on the project page, 'This module does not meet the standards required by the security team.' would warn off anyone landing on the project page from Google too (though I'd wager most module searches are done through d.o.).

[...] under the current rules each contributor at least goes through some kind of an approval process before they can unleash their code on the world and the security team is responsible for it.

In my opinion that approval process can be automated. If a module has: tests, a stable release, passes automated Coder.module checks, etc. it is ready to be supported by the security team and enter -- at the bottom of -- the d.o. module search results.

Three advantages to this approach:

  1. the approval process is not just applied once, but over-and-over;
  2. the hard-work is writing the code for ranking modules, instead of reviewing an endless stream of account requests (guess which is more fun!);
  3. duplicate modules will have a lot of work to get near established modules in security team support and search listings, but it's still possible;

Having said all that, I think we should do #1 first, then work on this later. The git move and this policy seem like entirely different problems. :)

I love your response to my

Dave Reid's picture

I love your response to my polite request in a module to explain its differences with an existing module ( is that I was on a "duplicate module witchhunt". Yeah.

Where in the bloody hell did we ever say doesn't want your code? That's what the entire purpose of the git sandboxes are for. We want you to share it.

Senior Drupal Developer for Lullabot | | @davereid

tim.plunkett's picture

As someone who got their CVS account as a co-maintainer, I was surprised to find that there was no review process to start making new modules. I never went through a single code review to get my account.

In addition, those that get CVS accounts for a theme have not demonstrated any ability to write modules. Similarly, just because you understand module development doesn't mean you have the know-how to create a theme.

I'm not advocating for more distinctions to be made between themes and modules, just that getting one project approved shouldn't result in complete free rein in contrib.

Same boat

ergonlogic's picture

I too bypassed a CVS account code review, via the co-maintainer loophole. In retrospect, I suspect I would have benefitted from having had to go through one.

I don't see any a priori problem with distinguishing approvals b/w modules and themes (and profiles, features, etc., for that matter). They have sufficiently distinct skillsets to warrant discussion, at least.

Option 1++

cleaver's picture

Moving to Git is going to make things much better to start with. I urge we take an incremental approach, then see what the impact is, then and only then decide if something more needs to be done.

2 just seems like so much work. Earlier this year I went into the CVS application queue and reviewed a couple of applications that had been sitting around for months. In both cases, the once eager contributors had become frustrated and ultimately abandoned their applications.

I can only imagine what it would be like if every single module needed approval. Sure merlinofchaos would have to be approved, but I guarantee he would get someone to review and approve his project much quicker that the rank and file. We want people to contribute to Drupal, not to drive them away.

3 creates more barriers to becoming a contributor. I think enough people are put off by the "rockstar" business and this would make it worse. We already value the heavy hitters and thank them every day when we use Views or CCK, etc. Let's encourage more people to attain the same level.

Option #2 wouldn't be as big

randallknutson's picture

Option #2 wouldn't be as big of a problem for me if the review was more about following the coding styles and security reviews and left out the politics of whether a module is needed or not. We need to make sure that remains a place (or becomes more of a place) where people can easily contribute their ideas and not be yelled down that their idea is bad. There are already many flamewars going on about forking modules, overlapping functionality and such that would become much worse with option #2 and would probably lead many developers over to github.

As such I'd vote #1 until and if we find a better way.

Option #1

Zarabadoo's picture

No need to make this any more painful than we need to. As mentioned in the post, we can move toward other options. We should feel it out. Maybe there will be more options thought up once we all get settled in.

Like Option #1

jvandervort's picture

Keep things as they are. Yes, we have some cruft, but by adding sandbox projects we might have much less cruft. Incremental is the key! We can always add Option 3 later if we don't like things.


#1 - Painful but proven

michaelfavia's picture

While I totally agree with Daniel about the duplicated modules I also value the opportunity they create. I've been around drupal long enough to see the baton passed between different paradigms a few times (Flexinode/CCK, profile/Nodeprofile, blocks/panels) and I don't know thats its our place to tell people they HAVE to work with others. This is especially true when cant also guarantee the other persons willingness to work with them or share their goals.

I think what would result from a more draconian project review practice is that people would begin to host thier projects elsewhere (github, etc) instead of our centralized repository. Water takes the path of least resistance and a black market code repo is the next lowest barrier for most people.

I hate to see waste as much as the next developer but i also recognize the value of a fresh approach. While it has its own drawbacks it can and does lead to new and better ways to do things on many occasions.

With the above in mind, I see no reason why we can improve the way we present and select modules for display to the general public, listing related modules, describing HOW they relate and what differentiates them might go along way to reducing the "ugh, ill just build my own" solution while still encouraging people to try new approaches if necessary.

One thing this comment ignores is the increased freedom that might be afforded by using git based repos with regard to unsolicited or even unwanted collaboration. Will we be listing forks of the projects on the project page itself? Gits improved ability to merge changes between distributeds sources mitigates alot of the "they wont work with me or they dont share the same idea" issues above by letting people experiment with others in the open and have those changes merged back into the main source if the maintainer sees fit.

I think its best to follow our own philosophy of iterative improvement here and take up the duplication issue next on its own merits.


kvanderw's picture

"You cannot create releases of sandbox projects. The only way for users to get the code of a sandbox project is with git clone."

I am hoping this does NOT mean that you are removing the 'tag' component of git? I would still like to be able to tag specific points in the codebase. Even if you don't want to make a release from that tag.

Oh, and I would vote for the modified #3 (or #1 if #3 is dis-allowed).

Kurt Vanderwater
Drupal made Easy


Crell's picture

You can create whatever tags and branches within git you want. However, you will only be able to create a release node that points to a tag or branch with a very specific naming convention. And that is only possible once a project is marked "cool enough to have releases". The question here is the process for a project becoming cool enough. :-)

I'd sure hope not! I go with

Josh The Geek's picture

I'd sure hope not! I go with #1.

I go by Josh The Geek.

1 or dww's #3. But I am more

pivica's picture

1 or dww's #3. But I am more keen to #1, like somebody said, if it ain't broken then don't fix it. #1 is the most liberal approach and also if module is not good or not well maintained it will just die after some time.

I'm glad we are tackling this

Dustin Currie's picture

It's currently too hard to get in, but once you're in, there is no oversight and it becomes a hotbed of bad (and good) ideas.

As mentioned earlier in this thread, if oversight is too stringent, the ecosystem will starve. And nowadays, isn't the only option to host code. There are a ton of code communities, with Github being the most notable one. Anything that makes it harder to contribute will have continually increasingly negative effect on d.o. contributions as social coding sites continue to gain popularity.

I'm a bit intrigued by #2, but let me throw out a big spin on it for discussion's sake: Let anyone and everyone create a project but have the community maintain well curated lists of projects. Perhaps there can be a community tag system for module searching and ordering. "This project release passes coder," "This project was last audited by a security team member on xx/xx/xx", "This project is in use on over 1,000 sites" and maybe last but not least, "This project has received overwhelming community approval." Something like a gold star.

That way anyone can contribute, the barrier to entry becomes nearly nil, but module usage and community eyeballs will cause the wheat to separate from the chaff.

Duplicate modules, bad ideas and sub-standard projects wouldn't be highlighted and a variety of factors, not just the judgment of a couple of the right people will cause the project to be tagged in a way that it gets shown in default searches.

These tags, or badges would be welcome and useful identifiers to newcomers and evaluators. if you stick to the community curated modules, you would be significantly less likely to run into bad projects and duplicate projects than you would now.

This idea embodies RTBC values and keeps #1

FatherShawn's picture

I think that Dustin is on to something here. We could migrate as the Git Team proposes and follow their recommendation to use option #1. We could then build a variety of ways that projects are "reviewed and tested by the community" - Dustin's categories are a good start.

We accomplish one change - move to git. Then we work on ways of improving an already highly functional culture. As a new maintainer, I would be motivated to work hard to get those badges or endorsements. We could clearly lay out expectations for each endorsement, which creates an affirmative culture for developers in addition to the benefits to end-users that Dustin articulates.

Drupaling in NYC since 2009!

I was just about to post

caschbre's picture

I was just about to post something similar... glad I read through this long thread. :-)

I'm not a fan of limiting access for folks to add projects. They'll just go elsewhere to host them which I think is bad. When I look at modules I'm more interested in whether they've (both module and owner) passed a certain criteria. I'd like to know whether the owner responds to requests in all of their projects. I like to know how often they commit updates. I like to know whether the module is technically sound. The list goes on and I'm sure we could add more 'criteria'.

I think in the end I'm looking more for stamps of approval on projects rather than trying to keep folks out. I'd rather see someone new to creating projects stumble and get help on d.o. rather than be turned away and have to go elsewhere.


Josh The Geek's picture


I like it the way it is, but I think that Michelle could use some help in the applications queue.

I go by Josh The Geek.

I would have much more

Dave Reid's picture

I would have much more interest in reviewing applicants/projects/code if I could see them once they've gotten used how to commit code to, use coding style, and have a place where I can review them easily (and not having to download random .zip files), which is why I'm for project-level reviews. And even a long-time contributor can forget to search and accidentally create a duplicate module. I'd be ok with my own new projects undergoing a review.

Senior Drupal Developer for Lullabot | | @davereid

I'd like to see the review

catch's picture

I'd like to see the review process happen when the first release candidate or point release is made (this could include alpha/beta maybe). So people can get the initial git account just by filling in a form, create projects, have dev tarballs etc. Then when they want to release a release candidate or 1.0, the release node goes into a reviews queue.

This would mean no initial barrier to having the code on or most of the things that are available to contributors. But two things happen when you make a stable release that currently there is no oversight for:

  1. your code gets supported by the security team. So if there is ever going to be a good time to review the code, it's immediately before the first point release or rc is made. This doesn't me an the reviews of new projects need to be done by the sec team, most people can spot obvious XSS or database injection issues, and that's the majority of security announcements too.

  2. iirc the default listings of projects on d.o don't show projects that only have dev releases, so casual browsers of the project list will only start to see it after this point.

And as discussed in irc, I think this should happen for all contributors. So if I want to make a 1.0 release of it'd go through the same process as a brand new contributor. I've never actually made a 1.0 release of any of my contrib projects, and without looking through, I'd assume there are far fewer stable releases of new projects than there are successful cvs applications.

Oh and by the way, I've

catch's picture

Oh and by the way, I've never, not once, released a stable version of any Drupal contrib module. I have newish projects that I've started that have been in dev for months (partly D7, partly because I'm a crap maintainer), and I occasionally commit to contrib projects I have cvs access, and someone else creates the stable releases. That's about the limit of my use of Drupal cvs at the moment. So if this was in place, as a long term contributor, I personally would not have run into this review process at all. There are people at the exact opposite end of this scale, like merlinofchaos, sun, eaton - but I'd really wonder how many stable releases of brand new projects they post in a year.

option #4

catch's picture

I couldn't find the original thread where this was proposed, but had a further look and found it -, just to make it clear it's not my own idea, although I still think it's a good one. I've also pinged the security team mailing list to point people here.

Yep, I'm all for

Dave Reid's picture

Yep, I'm all for initial-release per-project reviews, and once that's over then the project is fine.

Senior Drupal Developer for Lullabot | | @davereid


dmitrig01's picture

I disagree with only approving the first point-release. This will cause more of the -alpha/beta-itis than we already have, which is a Bad Thing.

+1 option #1

rschwab's picture

I vote option #1. The git migration process should happen unabated by this separate topic. I like the idea of #2 and a lot of the ideas discussed above are worth exploring too, so lets do that separately and allow more time for everyone in the community to work on a consensus.

#3 ++

yktdan's picture

I believe I have a CVS account because I got on as a co-maintainer and then did nothing. I should not be permitted until I have at least contributed something that works.

Separate issue: Since sandboxes are not searchable, how do I find out that someone already has what I want? I could deal with a view that listed all sandbox projects with at least a last activity date, a major Drupal release number, a one-liner description and maybe a "I want this" vote column.

Option #1 please

winston's picture

I think many of the problems of project proliferation are being solved merely by the power and flexibility of both improvements to Drupal core and a variety of key modules.

And besides, once you are in a git world the culture of easy cloning to me seems to argue against any excess approval process. There will be a tipping point where you've made it just a bit too painful and off folks go to maintain their projects on github or wherever which I don't think the community wants.

Also, kudos on the sandbox project idea!


frankcarey's picture

With the option to re-discuss later. I would prefer to see more metrics / flags like our current "X sites have this module installed", or possibly "Passes drupal coding standards", by running releases through coder first, etc. And then better education on how to make decisions on modules instead of a top down approach.


Liam McDermott's picture

1 for now, though I can see the need for change in future.

The purpose of barriers are to stop forks, badly written or insecure code from confusing users who're looking for modules. It seems this is not a problem with version control so much as a problem with search and module listings. Project creation is a big deal due to tying projects to search results, without enough control over the rankings.

What about introducing ModuleRank™ a kind of radioactivity metric for modules, which takes into account:

  • passing automated Coder tests;
  • how many sites have the module installed;
  • whether the module has unit tests;
  • if it has been checked by a member of the security team;
  • if it has been reviewed in some way by someone who knows what they're doing;
  • has releases for supported versions of Drupal;
  • and so on.

Projects with a crappy ModuleRank™ are not included in search results or module listings (and may be set back to sandbox status after a period of time); ones with a low ModuleRank™ are placed low in search results and so on.

Ultimately I'm suggesting the human review should be replaced by an algorithm, with any user being allowed to able to create projects (with the caveat that creating a project shouldn't mean much).

Sorry if I'm barking up the wrong tree.

Not at all the wrong tree.

sdboyer's picture

Not at all the wrong tree. The more I read & think here, the more I think this type of approach is the ONLY hope. More, better metrics.

Arguing about letting projects 'in' is ALWAYS going to make people very unhappy, because it's arguing about the hypothetical possibility of something being great/horrible/duplicate. Even if there's already poor code there, it could always grow into something else as the author learns. It's a moving target.

We're a community that's chosen to keep our code in a centralized place. That gives us HUGE potential to slice, dice, and organize that code in ways that make it easier to decide which you want. We've taken some steps (esp. in the redesign), but honestly, we should be ashamed that we haven't done more of that.

Filter output, not input.

Interesting way to put it

Crell's picture

"Filter output, not input" is an interesting way to put it, and one I can get behind... However, the question at hand is what qualifies as output. Is moving from sandbox to "for reals" project an output? Or is searching for a module output?

That's really the crux of the question, isn't it?


sun's picture

First of all, thanks to @kyle_mathews for bringing up the wise and always right analogy/metaphor of input filters.

Drupal's principle has always been that way. I can not only get behind it - it also automatically incorporates all other, related comments on the topic in a natural way.

But indeed, @Crell's point is absolutely right: At which point does the output filtering start to happen?

  • Does it start with an official project namespace?
  • The first official release?
  • The first project by a certain contributor?
  • None at all?

I'd consider the new git sandbox projects as the (arbitrary) input. Everyone can create unlimited. Anywhere. But for the question at hand, we're interested in code/clones/forks on only.

So what's the output?

Daniel F. Kudwien
unleashed mind

However, the question at hand

Liam McDermott's picture

However, the question at hand is what qualifies as output. Is moving from sandbox to "for reals" project an output? Or is searching for a module output?

Agreed, this is the fundamental problem. The analogy I'm thinking of is the relationship between Google and web sites, which puts me in the, 'Searching is the output' camp.

A sandbox is like a private site you don't want indexed; checking the box to move a sandbox to a project is like opening the site for indexing. Of course, just doing that isn't enough to get you to the top of search engine rankings, so you have to do SEO too. The Drupal equivalent of SEO is making your code better and getting it used on more sites.

What excites me is a system like this could turn a barrier into an incentive to write better code. Also, if the ModuleRank™ algorithm is open for patches and discussion the complaints about an 'inner circle' and 'cabal' will go away.

(and, I get to make up really bad jokes about ModuleRank™ -- yay!)

Sandboxes should definitely

naught101's picture

Sandboxes should definitely be indexed, albeit separately from projects. Otherwise how is anyone going to know if they're duplicating existing code? People need to be able to discover potential modules as well as real modules. Of course, there should be some suitable, bright-red, annoyingly-hard-to-miss warning that sandboxes are not for general consumption.

I agree

kyle_mathews's picture

I agree. Someone's test code or not-quite-ready-to-be-a-project code might be the perfect basis for a new module you want to write or the solution to a sticky problem or lead you to a new co-maintainer for an existing project. Now that sandbox code can have its own homepage, issue queue, etc. they're a much more interesting and useful space than the previously.

Kyle Mathews

I really hate this comparison (apples/oranges)

Dave Reid's picture

This works good for Drupal's text format system since it assumes untrusted input. But people that have the access to create projects are already in a sense trusted.

Senior Drupal Developer for Lullabot | | @davereid

Option 1

jbrauer's picture

Option 1 is a clear winner to me. We've come this far (Drupal 7) and it just works. Are there occasional issues. Sure there are. But these are handled in the proper way as well - through the existing issue queues. We don't need to fix what's not broken.

The other suggestions get into reviews that are destined to be more political and don't, at the end of many long debates, really solve the problem. Each of us can certainly point to situations where established maintainers, who would certainly have full access in #3 post make full projects that others question for any number of reasons. The beauty of the community is we have forums for handling these questions and having the discussion. At the end of the day it works out -- the wonder of a do-ocracy.

Also I hope we could come up with a way to search sandboxes. Not offering this level of collaboration, whether on a problem, a bit of a theme or something that might become a full module leads to either duplication or somebody doing the indexing/cataloging off of d.o and ultimately diluting the value of our community hub/resource we've spent so much time improving.


sun's picture

The other suggestions get into reviews that are destined to be more political and don't, at the end of many long debates, really solve the problem.

You would be surprised of how many CVS applicants were very grateful for letting them know that their suggested code/project already exists, when I was still allowed to do such remarks in CVS application reviews.

The beauty of the community is we have forums for handling these questions and having the discussion.

With a very high confidence, I can tell that almost all of the "more prolific" contributors never look into the forums. What you essentially get there are smattering recommendations.

At the end of the day it works out -- the wonder of a do-ocracy.

The logic/structure that is being proposed with #1 does not work out for me. For example, as an end-user on github (not actively using it as a developer), I have no idea whether I'm currently looking at someone's stupid unofficial fork of some real project or whether I'm looking at an official project.

It's cool that everyone can throw out his/her code to their likings, but no, end-users and other developers are only interested in the leading projects. We don't need too easy forks, and neither duplication of projects, nor duplication of efforts.

Please bear in mind that everyone on this thread is looking at this topic from an insider perspective. Right now, achieves the high amount of collaboration and contributions by making sure that insiders are providing tools for outsiders. Outsiders are slowly turning into insiders by collaborating and contributing their improvements to the projects they care for. Because they have no other way. Because they would need to take their code/fork and go through a application review process. If they find that too challenging, then they may head off and consider github and whatnot instead. But no one cares about what kind of modules may exist on github and possibly elsewhere, because no one knows of that code.

Likewise, I expect that no one will care for what code exists in sandboxes already (just like now). However, arbitrary code in sandboxes is not intended for general mass-consumption. It's sufficient to have a fully-fledged project infrastructure along with your sandbox, but do I really have to bug countless of end-users with that code?

And if I'd intend to publish that code for end-users and think it's right and it doesn't exist yet, then why should I remotely fear a project review?

What's the effin' problem?

Daniel F. Kudwien
unleashed mind


gapple's picture

I think a couple valuable points were brought up in support of dww's #3, where new and trusted contributers have separate permissions.

tim.plunkett mentioned that co-maintainers are given full access to create modules without the same review process as a brand new contributor: it is assumed that the maintainer that supported them did a sufficient review but there is little guarantee (e.g. a new contributer grants an even newer contributor access because they need the help and it was offered).
Several people have also mentioned that many contributors are sufficiently trusted by the community to create quality projects. I think it would be frustrating for a new contributor to view another project approval application that has been approved with little comment just because of who the author is, while their application is sitting in the queue unreviewed or with thorough criticism; they may not understand the history and reputation of the other contributor and feel slighted.

As Crell said though, the criteria for promotion (and demotion) must be clearly defined. Another concern is that the process for subsequent applications isn't an additional deterrent for contributions; it should be possible to say "you've already passed the hard part, we just want to check over things quickly" so that they don't decide another application is just too much work.

For subsequent applications, reviewers can focus on the areas that were issues for the initial application, hopefully speeding up the process. A valuable test is then available for granting someone un-reviewed module creation privileges: have they created a new project without repeating their original mistakes? Have they sufficiently supported their previous module, and continued to improve upon any issues brought up in their original review? If their subsequent applications go through with little comment, it is likely that their future efforts will be of equal quality.
This may not be at the same point for all reviewers. While it may be a good idea to require at least 2 successful applications to achieve trusted status, some contributors could take much longer if their first efforts are consistently incomplete (and then be encouraged to develop ideas in the sandbox further before applying).

mlncn's picture

I think we must do #2 because it forces all of us to make this process non-Hellish. The more i see some of our best module developers say, in effect, "do not ever put me through that", the more i feel we must all face the same review process if we are to make and keep it a good process.

As long as only new people, or relatively new people, have to go through the review process, it will fall into forgotten horribleness.

If we in the community now can't agree to go through a process for the first release of every project, how do we ask new people to go through the process?

With a focus on:

  • security review
  • non-duplication

There's not much justification for exempting anybody. These things can always use another look.

We can make this really simple— anyone granted a trusted status (so we still have need for much of the decisionmaking being made in #3) can promote any project except their own. The person who approves it is recorded and so face public embarrassment if security issues quickly arise or projects already exist.

If we're talking about contributing code to the community, getting at least one person (or two or three people with a reviewed/RTBCd/committed approach) to look and sign off on it seems reasonable.

benjamin, agaric

There are other solutions

mikey_p's picture

There are other solutions than since it sucks, everyone should have to have to go through it? That's terrible logic. Why not actually work on making everything easier? That's exactly what both #1 and #3 are proposing.

It's not like the developers that are saying we shouldn't do that, haven't been through the process themselves, on the contrary, I'm pretty sure they have and want to change, not make everyone else suffer.

I am not saying make it bad for everyone

mlncn's picture

IF only new people have to go through with it, it WILL be bad. No matter our intentions, our attention will not be there.

If we all go through the process, we can make it good as a community.

benjamin, agaric

First: Yay sandbox. This

Mile23's picture

First: Yay sandbox. This means I can put my github drupal stuff here, where it makes sense for it to be, so that everyone can pick it apart and put it back together as they need.

Second: Projects should be on a code basis, not a user basis. That is, there should be no access setting for you to automatically promote your own projects beyond sandbox. The project should stand by itself. It might work like this:

1) You add at least one co-maintainer to your sandbox project. All contrib should have co-maintainers before they're contrib.

2) Add an issue of a new type, something like 'request contrib status,' where you plead your case in public. Only one of these issues is allowed per project.

3) In the subsequent thread, anyone can change it to 'deny contrib status' and prevent you from requesting it again for a month (or some reasonable amount of time). I could say: "Your module doesn't have a .test file.... denied."

4) In the thread, any non-maintainer can give you an 'approve contrib status.' If you get some number of these without any denies for a month, you're in the queue for an abbreviated approval process, making it much easier on the set of people tasked with that queue.

5) The final approval/denial must also be given in public on the issue thread, and if they deny the project, the process can be re-started with another 'request contrib status.'

This is a peer review by actual peers, in public. Everyone learns coding standards, security concerns, and dev tools just from reading the process thread.

This takes a lot of pressure off those with the power to approve or deny the project, because it's clear from the thread if there are problems or not. This reduces the possibility of denials seeming to be political or somehow wrongheaded.

Someone who's a Drupal personality would still have to go through this. Their reputation might make it easier to find all the people involved to vote it the thumbs-up, and might intimidate deny trolls away, but that's not a problem because reputation should count for something.

So this is my vote for a modified Option #2, which would no doubt involve a great deal of coding. From a practical standpoint, it's probably better to go with Option #1 for the time being and revisit this after the git infrastructure is in place.


bleen's picture

If #2 were in place I fear that a module like admin would never exist in a world where admin_menu already does. Admin has evolved to serve (some) different purposes and admin_menu still has some features not covered by admin...

We must err on the side of encouraging people to submit code as much as possible even if that results in some code being duplicate or crufty.

That's my story, and I'm stickin to it...

Option 2++

gopherspidey's picture

I would be for option 2 if you could create releases and search. What is the point of restricting those options? I would just make it very clear that it is a sandbox and not search them by default.

There is enough crappy modules or unmaintained modules already in the namespace.

If you would allow searching and releases, then all we are talking about is protecting the namespace and helping newbies out by cleaning up the junk in the sanctioned projects.

The git migration should

drumm's picture

The git migration should limit scope to migrating to git, not changing policies along the way.

Which is why I ultimately

sdboyer's picture

Which is why I ultimately gave my nod to #1 as our official idea - it's the closest to our current system, and our mandate is for a 1:1 conversion. However, you're also oversimplifying - git isn't just a slot-in replacement for CVS. Differences in CVS vs. git dictate that some things, like how account approval works, must change. But since that does take us outside a clear 1:1 conversion scope, we need to come back to the community, as we have with this post, to present the situation for discussion. And, since we're presenting it for discussion (and we're unlikely to be able to drum up such widespread interest post-migration), it only makes sense we talk about the best possible solution, and see if we can't launch with it. Might as well use the momentum while we have it.

I should mention that formal

drumm's picture

I should mention that formal review is a huge task. Here are the official ones for the last 100 days Some are truncated off the right side.

We do need more ways of evaluating projects, formal, informal, qualitative, and quantitative, but that is a different project.

Unfair and out of context statistics

Dave Reid's picture

To be fair, that is every release made in a day, not just the first release of a project, which would be significantly less.

Senior Drupal Developer for Lullabot | | @davereid

Yes, that was a really quick

drumm's picture

Yes, that was a really quick query. There are 4362 projects with releases following the same query. They have 20348 releases. Or, 4.66 official releases per project. The actual burden would be approximately 1/5 of that query.

(Disclaimers - really quick query, changes in process may change developer behavior)

The proposal to review

catch's picture

The proposal to review releases, is not to review every release of every project, it's to review the first release of new projects. So that image needs to be put in perspective - the vast majority of those releases are not 1.0, and even some that are look like they are Drupal 7 ports.

Ok, so after a big discussion

Dave Reid's picture

Ok, so after a big discussion today in IRC, I move in favor of #1 only due to time/budget for Git migration. We can't change the process at this time. I'm still strongly in favor of a per-project review that happens when you want to create releases and give your project an official namespace.

Senior Drupal Developer for Lullabot | | @davereid

For the git migration launch,

arianek's picture

For the git migration launch, I fully support maintaining the status quo, ie. #1

If the plan of providing some new features on the project pages (reviews, five star, etc.) comes to fruition, then I would support maintaining #1. (It is nice as far as lowering barriers to entry, and with the sandboxes, I would support being more lenient on initial reviews.)

That said, I have real concerns about the continued cruftiness and duplication of projects in contrib. So if there wasn't a good rallying to get/keep contrib cleaned up, then I would fall back to #2, where each project needs to be reviewed before an official release. (Yes, I understand all the arguments against this, but I believe it could be a supportive, collaborative process with some fairly major changes and more people reviewing, which obviously has always been a hopefully 1 will pan out with the added features.)

I generally dislike #3 because I think creating arbitrary tiers of contributors might be ridden with fail.

Stay focused

zzolo's picture

Congratulations and awesome work on making all this happen. It's a huge, important step for the Drupal community.

What I am hearing (and liking and is initially proposed) so far is that #1 is probably the way to go at first, but that some version of #2 or #3 should be implemented in the near future. Please note that, IMO, #2 and #3 are the same just with a different value for the number of approved releases variable.

So, I do just want to iterate, please try to focus comments more on what is the best for the initial migration to Git, hopefully, in Feb. This means that we should not discuss the "ideal" here, but the practical given time and abilities. And comments for the future are important, but as @drumm points out are not really in the scope of the Git Migration, so should be minimized here, and maybe a full discussion can be started somewhere else.

+1 For Option #3

mermentau's picture

The sandbox idea really gets me pumped. I'm working on my first module over at github and I can bring it home.

OK, generally speaking I'm

EclipseGc's picture

OK, generally speaking I'm "FOR" option 1. With that said... I see the cruft it has led to and could also get behind option 3. Option 2 seems entirely too ambitious, and too draconian. Obviously the main issue with # 3 is the definition for how a user moves from one role to another.

As a stop gap measure, since this is ONLY a configuration issue, I believe we should opt for option 1 for the moment, and then re-evaluate the realities of our situation once we have a working git migration completed. The bottom line is that we can revisit this issue if we truly feel it's problematic, and we're not any worse off than the current method.


Heck no to #2

webchick's picture

(and, by proxy, #3)

[Note: This is an extremely opinionated rant by webchick, the community member. I'll post later as webchick, the Git migration team member, and it will be an objective summary of the discussion at hand for those who can't keep up with 80+ comments.]

Too much change, too suddenly

Contributors are already going to be dealing with three major shifts as a result of this move:

  1. Re-learning how to maintain a project all over again (copy/pasting new commands, learning that 'git checkout' means an entirely different thing than 'cvs checkout', getting themselves out of totally new scary error messages when things go wrong...)

  2. Coming to terms with the idea that anyone at all with a account who checks a box can now contribute code, and how that will ultimately impact the community and contribution workflow.

  3. Coming to grips with the contribution landscape now that there are "sandbox" projects, and how those will ultimately impact the community and contribution workflow.

So piling yet another change that has profound community and cultural implications seems like an enormous risk to me.

We can't keep up with the volume

New CVS application roll in at a rate of about 2-5 per day. The oldest ones have been sitting there waiting for approval for over 11 weeks. In general, people only approve applications when they feel they have some expertise over what the module is about or if they know the person personally. Reviewing is hard work.

New modules, on the other hand, come in at a far more rapid pace. There are 11 so far today, 5 yesterday... and this is a holiday week. We could work up a backlog of module approvals much grater than the number in the CVS application backlog within about 3 weeks.

So who are these mythical people who are going to review and approve these modules as fast as they come in, and why aren't they currently helping with our CVS application backlog right now, where we so desperately need them?

We need to remove barriers to contribution, not introduce more of them

Contributors are the lifeblood of our project. Without them, we have no Drupal. We have therefore bent over backwards to make it as easy as having a account + some basic tools where necessary (diff, patch) to contribute to almost all aspects of the project: core, documentation, translations, participating local user group, issue queue triage, patch reviews, patch contribution, test authoring, etc. This is a good and happy thing.

To contribute new code, on the other hand, you currently need to jump through these hoops:

  • not just a semi-working, but "release ready" module/theme
  • scouring of all projects to make sure module/theme doesn't exist yet (and hope it didn't spring into existence while you were making said release-ready module/theme)
  • an in-depth understanding of coding standards
  • an in-depth understanding of Drupal APIs
  • an in-depth understanding of how to write secure code
  • application form submission
  • tar or zip
  • an abundance patience to wait for an incredibly busy volunteer to respond (which, btw, might never actually happen)
  • incredibly thick skin to take the comments of whoever reviews your application non-personally.
  • more abundance patience to wait for a volunteer to respond again so you can repeat the process again a few times.
  • irc (to raise the fact that no one responded to your application once you'd finally resolved all the issues)
    and finally, if you're very, very lucky...
  • cvs

This makes absolutely no sense to me. People write crappy documentation, contribute incorrect translations, and submit half-baked patches. But that's great, and we welcome it. Why? Because it gets peoples' foot in the door of our wonderful community, and because even the most "needs work" attempt at something can be corrected, and improved upon by others.

So ideally, I'd want contributing new code to be the same as everything else on d.o which is: "Just Do It!" However, the security team has threatened to kill me. ;) So therefore, I opt for the next best thing, which is only force people to jump through hoops once, and then never again. (Option #1)

In the age of and where 0.01% of users contribute to the project, we can't afford to throw unnecessary barriers in front of contributors. If we do, we will lose as our central collaboration hub, which has been our project's single biggest strength for its past decade of existence.

Duplicate projects, crufty projects, badly coded projects, etc. are legitimate concerns but should not be tied to granting contributors access

Why? Because there are many other ways of dealing with those issues:

  • Reviews/ratings on project nodes (already slated for a future phase of d.o redesign)
  • "Badges" on project nodes to denote test coverage, coder module compliance, etc. as a quality indicator
  • A feed of new sandbox projects, which can be reviewed by the folks who can chime in as things come in before a bunch of code is written and the project is ready for approval.
  • "Opt-in" code reviews to request feedback from the folks.

In other words, we should be adding metrics to in order to help the good projects rise to the top, not dragging down enthusiastic contributors with rules.

I'm sure I have more to say on this topic, but that's enough for now.

I don't think we should have

catch's picture

I don't think we should have reviews of new modules, but I do think we should have reviews of the first stable release of new modules. By definition, since not every module gets to the point of having a stable release (don't know where to find stats of this but it'd be useful), it has to be less than the new modules posted, possibly less than cvs applications too.

DamienMcKenna's picture

Before a module/theme gets to the status where the security team have official responsibility for a module's first release (RC? I forget), there needs to be a review to check against the security standards, at the very least. This kind-of fits with the Option #4 suggested earlier by @DaveReid and @catch.

The natural consequence of this...

webchick's picture

The number of modules with stable point releases will drop dramatically.

This will leave users in a quandry being forced to deploy alpha/beta code to their sites or not use Drupal.

Those modules where alpha/beta means "I don't feel like dealing with politics" will be indistinguishable to those where alpha/beta means "will blow up your site".

We'll get a ton of pushback saying all Drupal modules are buggy and horrible, etc.

So I don't think this is a good idea, culturally, though I understand logically why it makes sense.

It would be impossible for

catch's picture

It would be impossible for the number of releases to drop, since this would only apply to brand new projects.

It's possible that the proportion of modules with stable releases might drop (do we even have that statistic available now?), but I'd rather have a tonne of dev projects on, than a tonne of dev projects floating around on github.

Even if it turned out to be a problem, it's much better than having the barrier to contribution earlier in the game, or not having any review of projects before they're "supported" by the security team - every, single, project with a stable release is potentially hours of work co-ordinating an SA so I'd have no issue if there's a few less new ones all the time.

Automated testing

DamienMcKenna's picture

How about having automated Coder and Coder Tough Love tests for all releases? It could then show the test status beside the download links so visitors would know up-front whether the code was at least clean? An extension of that would be for maintainers to be able to trigger Coder / CTL tests at any time for their codebase, without having to build an actual release.

Another way of focusing on

sdboyer's picture

Another way of focusing on filtering output, not input. +1.

Mentally re-titles this post:

cleaver's picture

Mentally re-titles this post: "Sloppy Sam applies for project creator status"


neokrish's picture

Duplicate projects, crufty projects, badly coded projects, etc. are legitimate concerns but should not be tied to granting contributors access

Why? Because there are many other ways of dealing with those issues:

Well said!

This bears repeating

jbrauer's picture

In the age of and where 0.01% of users contribute to the project, we can't afford to throw unnecessary barriers in front of contributors. If we do, we will lose as our central collaboration hub, which has been our project's single biggest strength for its past decade of existence.

2 will lead to more of our community going elsewhere.

3 with it's inherent inequality will send them away even faster.

+1 for option #1

levelos's picture

Aside from the obvious inability to effectively review new modules by an ever changing group of volunteers, a review wall would simply add to the growing hierarchy within the community.

Lev Tsypin

ThinkShout, Inc. |

#1 until #3 can be implemented

tnanek's picture

I like the sound of option #1 until we have the infrastructure to support #3.

3 needs to have a unchanging number for N. In terms of what needs to be there for an official project (for those in group A), a security review of those may be in order, as well as a coding standards review, but beyond that, the reviewing of the module should be left to potential users of the module once it starts to be used; those in group B have been around for sometime and have passed a number of checks of this sort already, so no need to keep verifying that they know what they're doing.

The coding standards review could be automated through coder (supports drush, so it shouldn't be so hard to implement). Thus the security review is the only thing to really consider. Though this may increase the delay, work con continue as it is sand-boxed, and it would help make sure that modules which are showing up in the searches are safe for the standard person to use (or at least were at one point).

As for the infrastructure for it, we'd need the automated script for determining coding standards. There already is a security team which reviews security related submissions about modules, essentially we'd be appending to their To Do list, but this would help prevent future security risks, so I don't see it as being too troublesome for them.

Summary so far...

webchick's picture

Here's a summary of the 80+ comments so far:

Weighing the options

There were several concerns raised with keeping the status quo (Option #1):

  • The contributions repository is filled with crufty, duplicate projects and poor-quality, difficult to understand code that lacks documentation.
  • The Drupal project has reached a scale and momentum where there almost always IS a module for that, and one-shot projects and duplicate efforts no longer have a place.
  • Sandbox projects opens the door for forking, which means more duplication and less collaboration. Not placing controls around this means we could end up with chaos.
  • Since oversight is only done at the beginning, contributors still create duplicate projects later on, which leads to end user frustration/confusion and wasted effort.
  • The ability to create themes does not equate to the ability to create modules and vice-versa. Right now, it's "all or nothing" access which is incorrect.
  • Even long-time contributors can make mistakes in their modules, and would benefit from someone looking over their shoulder before publishing a project to the wider community.
  • The effect #2 would have on an average contributor is overblown. Most people do not create more than a couple of new projects a year.
  • There are several ways to "hack" the existing approval process, e.g. become a co-maintainer. It's also a process that people only need to go through once, after which they're in the "in" crowd. This creates an air of unfairness that would be addressed by forcing all contributors to be in the same boat.

However, there were also several concerns raised with introducing more oversight (Option #2):

  • Competition is seen as a valuable thing which allows innovations to occur like Admin module alongside Admin Menu, Ubercart alongside E-Commerce, etc. which likely would have been denied had their respective authors needed to go through an approval process at the time.
  • The decision of what modules to use should be in control of the community and our users, not admins. Good modules will naturally float to the top, crappy modules will naturally sink to the bottom.
  • This option prompted statements like "will break the community" and "our long history of having a rich and diverse module/theme repository would be put in jeopardy." It's a major cultural shift, and we're safer to go about change in an incremental way.
  • The existing approval process is already keeping out major contributors and tons of code, even though it's a one-time thing, because of the politics people feel they have to endure. Introducing an even more stringent approval processes will drive even more people to host their code off-site or not share it at all.
  • Git brings with it a culture of easy cloning, and fighting against this is a losing battle.
  • It's not in scope for the Git migration (a technology issue) to impact policy (a community issue).
  • There are serious reservations about reviewers being able to keep up with the volume of new projects coming in.
  • The complaints people have with the existing "free-for-all" process can largely be mitigated with better searching and filtering tools on Several options were suggested here, including community tagging, security team audit indicators, community approval ratings, badges for test coverage + coding standards, user reviews, a ModuleRank™ overall auto-generated rating, etc.


  • The idea of "sandbox" projects looks like it has broad support. A couple of people mentioned that they can now move their github code back "home" to, which is awesome and exactly what was intended. Yay! :D

  • On the flip-side, it was pointed out that promoting sandboxes to a "full" project status introduces the possibility for user confusion as to what the "real" Views module is as opposed to someone's fork where they're playing with something. It also likely increases the chances of intentionally forked projects because of disagreements with maintainers or impatience. This breaks "collaboration over competition", which is a fundamental community tenant, and currently basically a requirement because there is only one canonical Views module, for example, and doesn't appear anywhere on

  • On the flip-flip side ;) while forking is easier to do with Git, so is merging in forked changes.

  • There were some concerns regarding the findability of sandboxes which could use more discussion.


A few counter-proposals to the two mentioned in the original post have been suggested:

dww proposes a middle-ground Option #3, where all code contributors default to having each new "real" project approved (Option #2), with an option to apply for being exempt from the checks for long-term contributors who are known to post consistently good stuff. This has pretty wide support, as well as some detractors who worry about the "two-tiered" contributor system it creates and how to keep that fair.

catch proposes moving the approval process from "real" project creation to instead approve the creation of the first RC or point release of a module (at which point it becomes the security team's problem). Let's call this option #4. This would result in more security oversight than currently exidsts, without impacting every contributor and project.

mdekkers proposes an Option #0, which removes any and all restrictions around committing code and welcomes module duplication, abandoned modules, etc.

Dustin Currie proposes option #1, but modified with the addition of a community-maintained curated list of awesome projects. Something akin to "Golden Contrib" which is an idea that's been kicked around for a number of years. Let's call this option #5.


In terms of raw vote numbers (which is a terrible basis upon which to make a decision, but good for spotting "trends" nonetheless…), here's how these shake out currently:

Option #0: 1
Option #1: 29
Option #2: 5
Option #3: 10
Option #4: 1
Option #5: 2

(here's how I counted those)

Option #1 on the surface appears to have a ton of votes, but it's worth pointing out that some of those want the ability to re-visit the decision post-launch once we see how things shake out a bit. Option #2 appears to lose miserably, but when coupled with #3 (which is really just a modified #2) the numbers are fairly strong, though still only about half.


Given that there's not overwhelming consensus to go with #2/#3, and some serious concerns expressed by folks about introducing more approval/oversight, #1 still appears the safest way for us to go for initial launch.

However, this discussion is very good because it's bringing to light a bunch of frustrations and ideas that we can try and flesh out more for the future. So please, let's continue. :)


Josh The Geek's picture

Let's create a poll for this: Option 1, 2, 3, 4, 5, Modified #1

I go by Josh The Geek.

Label forks

DamienMcKenna's picture

I think a key problem with forks on Github is the lack of appropriate indicators on the project page in large blinking marque'd red writing that "this is a fork of [otherproject]", so adding that would greatly help. Maybe if there was a way to require a project fork have a "rationale" field to explain why the fork was built, it would also help visitors (and the original module's developers) identify why it is available.

I'm pretty sure that's on

kyle_mathews's picture

I'm pretty sure that's on purpose because what if one fork does take off and because the default "best" fork? How can you pronounce who the eventual winner is at the second a project is created?

That's why they provide the Network chart and other indicators to see who's doing what. Programming is irreducibly complex especially open source programming and we're never going to reduce the complexity of using Drupal to shrink-wrapped software easy (there's commercial CMSs with their armies of support engineers for that).

Again, better metrics are needed not better gateways.

Kyle Mathews

Better metrics are needed,

sdboyer's picture

Better metrics are needed, not better gateways.


and an even better way of saying it than you did before.

And, by way of example, these are the scripts that actually do the migration process (pull all CVS data into git).

Note that it's a fork.

Shifted away from #2, supporting #1 for now

jurgenhaas's picture

Originally, when reading this post I supported #2 as I thought that Drupal is growing into a business class framework and needs a quality assurance process as much as it needs contributors.

However, after going through all the comments and webchick's summary I tend to support #1 because:

  • Too much change at a short period of time is dangerous
  • High quality code will be identified by "customers/consumers" anyway

So, for now option #1 is what's achievable and also what has been the objective of the "cvs to gethub migration" project.

Improving quality of community input should be a subsequent project and should consider all possible means like different access permission from options #2 and #3 as well as project tagging in a way, that the public (Drupal users) can somehow identify what's good quality code. Not quite sure yet how this should be done but at least some housekeeping process should be considered, e.g. remove (or sandbox) projects that are not actively maintained anymore or that obviously break coding or security standards.

#1 please. Permanently.

IceCreamYou's picture

I do not want to be a second-class citizen, under any circumstances. I do not want a super-special, secretive committee deciding whether I'm a big boy using an arbitrary process that depends on the weather on Mars that day and what they had for breakfast. I also do not want to have to be effectively "Drupal-certified" to be a first-class citizen. And I do not want yet another system that is painful for the majority of Drupal users, but which the "inner circle" doesn't care about because it doesn't affect them.

I don't care if we create a process to review every single new project (or its first release) or not, but if we do it, everyone needs to be subject to it. I don't think I should have to wait until someone gets around to reviewing my issue in order to create a new project, but that's "just" a logistical issue, assuming all such issues are treated equally.

I'm no fan of duplicate projects, particularly forks, but I'd much rather take action against them retroactively. A simple "flag as possible duplicate" button or something like that would suffice. Then the offending projects could be reviewed and moved (back) to sandbox status if necessary after giving the creator a chance to explain. Option #6, perhaps, although that's really more or less the way we do it now.

We often talk about how the Drupal community is great because anyone can contribute freely. Come for the software, stay for the community, right? We are one community, not a "trusted" community and an "untrusted" community. So a big thumbs down to #3, unless the promotion to the "trusted" role is automatic and based on a very low, very transparent threshold such as ownership of a single project that passes Coder and has been in existence for 3 months.

I support #1 but

dmitrig01's picture

I think it's quite unfair of you to say this:

super-special, secretive committee deciding whether I'm a big boy using an arbitrary process that depends on the weather on Mars that day and what they had for breakfast

Processes like this are not super-special and secretive: there's only one secretive committee within the Drupal community, and that's the security team, and for good reason. If a committee were deciding this, it would be out in the open.

This, however, could be automated: what if the metric was having five projects approved? Everyone would have to go through the same thing, no committees, and it depends on concrete metrics --- nothing arbitrary about it.


IceCreamYou's picture

The "secretive" part was hyperbole. I know that such a committee in the Drupal community would be unlikely to be secretive, and I didn't mean to imply that I think that such a group exists right now. I just meant that if humans are deciding who's "in" and who's "out," it's impossible to be wholly transparent because there will always be some amount of arbitrary decision-making about what counts as "quality."

On a not-really-related note, I also think that if we follow through with the whole "filter-on-output" thing, we need to be really careful not to heavily disadvantage new projects. Someone completely unknown in the Drupal community could come along tomorrow and post a module they've been tuning for years that creates the most amazing social network ever, for example, and we don't want that module to be relegated to oblivion just because the author is new and little-known. Any filtering on output should focus more on the individual project than on the maintainer (and after all, maintainers get better over time).

I fully support #1, and I

heyrocker's picture

I fully support #1, and I generally agree with all the arguments against #2 summarized by webchick above. Most specifically, I feel strongly that there are lots of ways to handle the problems of quality and duplication on d.o itself in terms of ranking/searching/filtering mechanisms, and I hope those projects start getting some traction (this discussion may even inspire me to dive into them.) I also strongly agree that the last thing we need right now is to raise barriers to contribution and open up even more avenues for people to feel that the Drupal community is filled with insiders and people just wanting to protect their turf.

That said I do also agree that all process questions should be under constant evaluation and open for re-jiggering later too, and it sounds like this will be easier than ever from a technical standpoint so great.

Reviewed by the community flag for releases

rayvaughn's picture

As a lighter alternative to #4 how about a review system for releases which is open to all contributors. Something like this:
I release 7--1 of my new project so I email a couple of other contributors I regularly work with and ask them to review the release. Once they have done so and are happy they tick a checkbox and the release is shown as "Reviewed by 2 people" on the module page, with a link to a page showing how many reviewers, who they are and comments. The security team audit can remain as it is - not required for a release - so that good code doesn't get delayed by an overworked team.

This would allow long-time contributors to utilise their standing in the community to bring in heavy weight reviewers, whilst providing a good way for high quality new contributions to quickly bubble up.

We could have a "Modules I reviewed" page as part of their profile, so if I want to find out what Views related modules merlinofchaos has reviewed I can find them easily.

Option #1, let's make it incremental

dixon_'s picture

Let's do the technical migration first, and then we can discuss the approval process.

I believe the most important thing is to quickly move over to Git and stop the community forking that happens on Github etc. Option #1 is the quickest path to get there. Once we've implemented Git on d.o and have less community forking, we can discuss the approval process. In general I agree with the most arguments from option #2, but let's think more about that later. We should keep focus on making the actual migration happen.

Merry Xmas everyone! Especially to the hard working D7 and Git migration teams! Thank you!

/dixon_, NodeOne

// Dick Olsson

They can't be decoupled,

sdboyer's picture

They can't be decoupled, unfortunately. The CVS application process has to be replaced by something, it simply doesn't generate valid data anymore. But yeah, #1 is the straightest conceptual path from where we are now (though it'll actually require a smidge more work).

I'd also agree with

juliangb's picture

I'd also agree with separating the Git migration from any changes to the renewal process, hence option #1 for now with a review of this as a separate process later.

Hopefully with increased visibility of sandboxes, more people will use them and hence make them more effective to work from.

My site:
Maintainer of: Drupal PM (Project Management).
You can contribute towards my Drupal work at

Module comparison

eigentor's picture

I am getting Webchicks point that the workload to review every new proposed module would be beyond reason. And also it is against the open spirit of Drupal.

To the outside world something else is important: how to find the best module for my task? We need to provide a lot more metrics/ratings/sorting criteria/comparison criteria on module listings and pages. It has gotten enormously better with the new d.o. and can still go a long way.

It can be 50.000 Modules as long as the most popular and best maintained are easy to find and it will harm no one.

Life is a journey, not a destination

I have to agree with my

solipsist's picture

I have to agree with my colleagues dixon and heyrocker. Option #1 is the way to go, it keeps the barrier low which is critical to the continued success of the contrib space at d.o. People are turning to GitHub because they feel the rules are too strict, and tons of quality code which should be committed to existing projects lie stranded on GH. It is better to think about how we can bring that code home and merge the various variations that exist or encourage modules to do their thing, and just their thing, really well. We're already on our way there through the change from monolithic to modular design, which encourages consistent interfaces and pluggability, clearly seen how modules like Views, VBO and Rules very often replace what used to be single purpose monolithic modules.

Also, with such a major shift in how contrib code is managed and maintained we need to make sure we're not biting off more than we can chew. The sandboxes are a wonderful new addition and we need to see how this change impacts contributions. Once we've seen that we can return to this question and see whether it is an issue, and whether we need to reconsider the policy.

Further, reviewing code takes time from experienced developers. As webchick pointed out, where's this ghost army of Drupal ninjas? As much as I prefer a proactive approach whenever possible, agile practices teach us to be flexibly reactive and I think it applies here as well. It's not just better, it's the only really feasible thing.

SLA for module approval

giorgio79's picture

If the new process will introduce some review by some voluntary Drupal team, it might end up like the Facebook app approval process, where you get to wait 2-3 weeks.

Perhaps an SLA would be welcome for such an approval.

If Drupal will grow to be 10% of the web, you will get dozens of new modules a day.

****Me and Drupal :)****
Clickbank IPN - Sell online or create a membership site with the largest affiliate network!
Review Critical - One of my sites

Seems to me that..

ilo's picture

.. poor-quality and duplicated modules is a different issue at all, so I'm going for #1 or #2 (this last I don't like it that much, but I agree it is a good filter).

I mean, the fact is that people is already creating duplicated modules to get a CVS account, because the CVS application asks them to provide a module to review. I would go to 'change CVS application' to first resolve/contribute to an issue in a module the user is using (probably any of the to 20 modules) and let the module maintainer vouche for his response or effort. Actually, most people is writting modules outside of d.o. regardless it being a duplicated or not, and some of them reach the CVS application process. This is IMO something that is affecting the whole review process and therefor Git as well, but I can't see how this will be changed by the proposals. #2 will only filter the modules listed in d.o., not preventing them from being created at all. It is not the first time a feature request gets lost because the module owner does not like it, and a module fork appears to handle it.

That is, making use of the git sandbox projects to clone and work on issues separately is a great win, using sandbox to create 'variations' of modules to include 'my single patch that never gets committed' can be a dramatic situation with, lets say, 300 different versions of 'Views', with different patches and goals. Now, being 'hidden' in d.o., I can't find a way for new users trying to attract attention to their modules.. they barely will reach the 'this module is a winner' status. New developers will get very frustrated because they might have done a very great work on something that nobody will look at. We are acting the wrong way I'd say.

And that underground category listing for 'views-2.x with patch to support flicker' (as long a extendr is not a module) will appear quickly, and will happend outside of d.o..

However, developers are still encouraged by the Drupal's self presenting word: 'framework', AKA, build your own, customize your way.. people is writting duplicate modules for some reason, and we (I'm doing it also, of course) are ignoring it at all.

Said that, I will gladly accept whatever is decided as an option.

+1 for option #1. That will

drecute's picture

+1 for option #1. That will surely encourage more contributors, hence, more modules and improved ideas which will definitely help in the growth of drupal powered projects

t: @charyorde

The more I think about it,

cweagans's picture

The more I think about it, the more I think we should go with option #1 for now and revisit the problem after we give CVS the boot. Too much change too quickly is going to be a Bad Thing, IMO. There's definitely a problem with the way that we do things currently, but specifically what that problem is and how we go about solving it seems like a discussion for another day.

Cameron Eagans

Yeah I think there needs to

catch's picture

Yeah I think there needs to be a 1.a/b option for this - I don't like the fact that we have 11 week old cvs applications in the queue, but there's no reason for this to block the git conversion - as long as things are left open to revise it later.

Clarifying my position

dww's picture

A) I proposed option #3 since that was one of the possible configurations that drove the development around sandbox projects in the first place. I do think it has some merit to consider if we're weighing all the options, but I now agree #1 is the way to go for the initial Git launch.

B) Module metrics, rating/reviews, and lots of other important features have been in the works for years. Many of them now exist (as one important example, the default project browsing sorts by usage, not A-Z). Unfortunately, when we were implementing the d.o redesign, these features were punted as "non-critical for launch", so although I snuck in a few more (e.g. projects with automated tests now tell you that), further work on metrics and reviews have been put on the back-burner. Then, the Git migration became the Next Big Fire Priority(tm), so we haven't been able to circle back around to finishing work on more metrics and ratings/reviews. Interested parties are invited to dive in and try to help. Good places to start are these two posts:

I'm not going to be able to keep up with every comment in this thread, so if you have an idea about possible metrics or ways that ratings/reviews would work, please post them to these other threads, otherwise, they'll probably be lost.

C) There's intentional duplication and unintentional duplication. Intentional, conscious, mindful duplication ("I have a fundamentally different solution to problem X than the existing solution Y, hence I've created Z because of A, B and C") is great (usually). Unintentional duplication is a problem, and I think the pro-competition crowd underestimates how hard this is for end users to sort out. People getting stuck on older versions of core because their site depends on a contrib module that was forked and no upgrade path was provided for them after a "better" fork took off. It's putting a huge burden on people with very little background or understanding to try to make important decisions that are going to affect them for potentially years to come. Even with vastly better tools to rank and find projects (see above), expecting people who don't know anything about Drupal yet to sort out and make informed decisions on a large collection of different metrics comparing 57 competing image gallery solutions is asking a lot of them. In spite of all of its (perhaps innumerable) failings, the existing CVS application has at least helped avoid some unintentional duplication, and that's good. I'm not saying "hurray for the status quo". I'm saying that even with better ways to find existing projects in the same problem space, it'd be good to have other ways to help avoid unintentional duplication. This isn't about developer insecurity, this is about not making Drupal itself unapproachable to site builders who don't have the benefit of a 150 person development team. I'm not saying a project review/approval process is the only way to address this, but I think it's irresponsible and unfair to say "this isn't a problem, and people who think it's a problem are insecure and have their own self-interests to protect."

D) Huge -1000 to Option #4 (review/approval to make non-alpha/beta releases). As others have already pointed out, it will only contribute to the existing problem of people rarely making official releases on d.o. If we put a big wall in place of that, it will effectively never happen. Plus, from a technical level, that'd require a bunch of additional coding and complication than the existing sandbox project code.

E) Will people please stop making insulting and ridiculous claims like "And I do not want yet another system that is painful for the majority of Drupal users, but which the "inner circle" doesn't care about because it doesn't affect them.". Every one of us working on the Git migration has been busting our asses trying to make this whole system no longer painful, and yet all of us probably qualify as part of an "inner circle". In fact, I've been advocating for deploying sandbox projects right now so we can just drain the backlog of CVS applications immediately by giving all of them permission to create sandboxes. But, that's not viable at this point for various reasons, so we're just going to roll out the sandbox projects when we roll out everything else.

Hey DWW, I don't

mdekkers's picture

Hey DWW, I don't underestimate at all how hard it is for users to pick the correct project, which is why I talk about creating better and more powerful systems to find and compare projects. I strongly believe that that is the way to solve the problem you discuss - not more manual approval processes and approval committees. I completely get your point about intentional/unintentional duplication of functionality, and believe me - I agree with that - we spend a lot of time reviewing modules for suitability on our projects - but again, how are you going to police those? what are the criteria? who is going to do the review of those, and most importantly: the current system clearly does nothing to stop this from happening and is known to be broken already.

On the flip side, I don't understand how the current system stops duplication (even a little bit) at all, as you seem to imply. Everybody makes a point about the 53 ways of doing an image gallery, but frankly, that is not going to go away, and drupal is big, versatile, and hugely powerful. the choices appear to be "keep it all cut down to minimal options, play to the lowest common denominator" or "lets make it easier to make sense of the madness, and let evolution take its course" - I think the latter makes more sense.


sun's picture

Everybody makes a point about the 53 ways of doing an image gallery, but frankly, that is not going to go away

Sorry, but that's BS. If I'd know with whom to collaborate, and in which project, then I'd eliminate all of the 53 modules within 4-8 weeks, solving my use-cases, your use-cases, everyone's use-cases.

But instead of encouraging more collaboration, the mindset seems to be "just happily keep on scratching your own itch, duplicate further and waste everyone's time".

If everyone prefers to waste time, that's fine with me. Not my world.

Daniel F. Kudwien
unleashed mind

Thank you, you highlight

mdekkers's picture

Thank you, you highlight exactly my point - you would eliminate all the diversity in the implementation and community to arrive at one massive "hivemind" solution to all problems. Well done you - now I have to pull this huge chunk of code into my site, probably consisting of a core API, some helper modules, and a bunch of tangential dependencies, and spend hours integrating, testing, and configuring this massive codebase, when all I want to do is show a couple of images in a nice little jquery thingy. I don't care if it solves the image display requirements of 10.000 other sites, I wouldn't even care if it solves the display requirements of some of my own other sites - in a great many cases, simple is best.

There is nothing wrong with collaboration, but not at the expense of all other options. That is when "collaboration" turns into "blinkered".

I collated the project

Liam McDermott's picture

I collated the project metrics mentioned in this thread in a comment on the 'Project metrics for redesign' thread mentioned by dww.

Hope that's the sort of thing you're looking for, Derek. :)


IceCreamYou's picture

Will people please stop making insulting and ridiculous claims like "And I do not want yet another system that is painful for the majority of Drupal users, but which the "inner circle" doesn't care about because it doesn't affect them."

I certainly don't feel this way about the Drupal community at present, and I don't mean to imply that anyone is intentionally excluding others. I apologize if you found that insulting. I only meant that allowing humans to make choices about who to give special status to inevitably creates a system that is not as meritocratic as I think it should be.

Few restrictions

deekayen's picture

I theorize people are hosting on github in part because there is no approval process to contribute or fork code. I favor the fewest number of approval hurdles.

+10 for Option #1

jpstrikesback's picture

Improving the existing working option (#1) with sandboxes is evolutionary, it enables Drupal contribution to grow and gives tools for better evaluation by those doing the approvals. In short, it's fantastic, cause it doesn't muck with a working ecology, even if the speed at which approvals happen is slow there's nothing to say that those approving applications cannot grow with the community. #1 is fertilizer.

Option #2 creates opportunity for politics and preference to undermine a meritocracy. Even if it creates an upside for code quality/security, that quality would be present in #1 anyways, as it is now. As Webchick stated above where would Ubercart be if it and E-Commerce had come out at the same time under #2...and from there where would Drupal Commerce be? Non-existant.

Largely I say #1 so that we leave module selection to the user, the uninitiated user innovates accidentally and outside the box, and will find novel and unconventional uses for Drupal as the framework grows more, and more flexible. Ratings systems, comparisons and even the fact that competing modules often join forces down the line will help with the issues created by a growing and largely unregulated community.

Cheers :)

+1 for Option 1

usonian's picture

For the initial switch, I think option 1 is the way to go. There are some very interesting ideas being brought up in the other options, but I think trying to switch to git AND roll out a major process change is a recipe for severe discontent at best and disaster at worst.

After the dust from the initial migration settles, then it would be interesting to have a larger discussion about how contributions on d.o. work, which would necessarily involve all of these questions about code review, duplication, metrics, et cetera. Personally, I think one of Drupal's greatest strengths is its centralized contrib codebase, and it would be too bad if the bar for contribution was raised so high that people bail and start rolling code on github or elsewhere... but I do think there should be a bar. (I really like the "This module passes coder" idea.)

Enhancing option 1

cleaver's picture

I will keep my support for option #1 or even option #0. I'm generally in favour of duplicate modules and letting them live or die through natural selection. However, there will always be room for enhancement. A few that I can think of:

  • automating the CVS/Git account approval process as much as possible.
  • automating Coder reports for modules.
  • don't require Coder compliance, but require maintainers to annotate any issues from Coder before they release.
  • encourage built in unit tests.

Maybe the best way to encourage quality is a point system based on: a) Coder compliance, b) security reviews, c) performance reviews, d) automated test coverage, e) user votes, f) issue queue stats, etc.

I think most of the above have been suggested, although I'm not sure I've seen automated test coverage, or performance reviews. I'd hesitate to include usage stats since there will be some niche modules with great quality that never have a really large installed base.

+1 to this. Let's increase

coderintherye's picture

+1 to this. Let's increase contribution quality through automated coder reports and testing instead of biased opinions.

Drupal evangelist.

Separation of initial migration vs post-migration

DamienMcKenna's picture

I think we need to separate this into two discussions:

  1. Continue this discussion on what is required for the initial Git migration, any project workflow adjustments that really can't be lived without otherwise major problems would be created. Clearly this would be option #1 with possibly some minor adjustments (see above for some ideas).
  2. Start a separate discussion on how to expand upon the initial release to further improve everyone's lives, both to not overburden the existing community with additional manual review processes but also to add indicators to help guide visitors towards more stable code & away from unstable code.

#1 for now make sense

pwolanin's picture

I agree with this - I think getting sandboxes and maintaining something like the current system will let us eliminate much of the CVS application backlog and get the git migration out of the gate and available to the world.

In the longer term I think we need better automated tools for code review, security review, and finding similar projects. A process for per-project approval to get a real short name, or to make initial releases that was more along the lines of mentoring combined with automated reviews might move us towards the higher standards we want.

Also, I agree that the git migration is a good opportunity to make as abandoned projects with no activity or releases since 4.7....

Security review stamp/seal

jvandervort's picture

Keep in mind too, a MODULE cannot be vetted by security review in perpetuity, only a RELEASE. In the future it might be nice if the security team could place a stamp/seal on a particular release. A "at this point in time (7.x-1.2) this module was secure." It would give the developer one more tool to maintain security.

After further review, I'm still in favor of #1:)
The Module quality assurance process should be divorced from the sandbox vs full project issue.


+1 to Option #1. However, I

coderintherye's picture

+1 to Option #1.

However, I would also really like to see all module project pages with no releases that have existed greater than a certain time length (i.e., there are project pages that have been up for years with no release) be removed from listings and search results, and relevant to this dicussion, have the commit access removed for those project maintainers if they never committed any code. After that is done, slowly work on moving abandoned modules out of search pages and listings and instead moved into a page of "Projects that need contributors". Then we could point new contributors who aren't sure where to start to there and say, hey, start here first by picking up maintainership on an abandoned module.

Or more succinctly, clean up the yard and exterior of the house that everyone sees before worrying about who has keys to get inside.

Drupal evangelist.

Step back

Crell's picture

I think we need to take a step back and consider what our goal here is. There are a lot of conflicting goals in play, and that colors everyone's perspective.

1) We want more warm bodies. More developers == good. This is taken rather axiomatically. However, just because this statement is part of the Drupal Faith doesn't make it true. Is it? Is simply increasing the numeric quantity of people who write code that lives on a good thing? If your answer is "Well of course, duh", then you are doing Drupal a disservice.

2) We want "good" modules, where good is some combination of "code that is fast, bug-free, security hole-free, and conforms to coding standards". Well, sure. But why? Because Drupal the product is not core. Drupal the product includes all code available from Security "firms" (sometimes legit, sometimes trolls) routinely refer to the number of SAs released against "Drupal", and include all contrib SAs. When we talk about the strengths of Drupal, anyone who knows anything quickly points out that core is useless without contrib. Extensibility is what makes Drupal worthwhile, and more importantly what it's extended with. Drupal 6 without Views and CCK is, to be perfectly honest, a complete waste of time as a CMS. The "official" contrib repository is part and parcel of Drupal the product, and it is on Drupal the product that the developer and professional reputations of all of us is based.

3) Because "Drupal the product" includes, we want the quality of the repository in toto to be high. Quantity of modules != quality of the repository. Having multiple modules that do the same thing and differ only because someone didn't apply a patch fast enough for someone else harms the overall quality of Drupal the product, because it reduces collaboration and increases confusion for users looking for modules.

4) Because "Drupal the product" includes, but not GitHub, we want the "good" modules to be on, not GitHub. My company produces multiple modules for most sites we build that are never released, mostly because they're not useful to anyone due to being too specific. More generic modules, though, are useful to more people and we try to release those. The corrolary to that is that yes, not all modules are equally good. Some are crap, some are useful to exactly one person, etc. Others are useful to nearly every site. The latter are the ones we want improving Drupal the Product. This is also why we have an SA process for all (released) modules on; they are part of Drupal the Product.

5) We want "good" developers, because "good" developers product "good" modules. Some built-in review step is, really, our only opportunity for more experienced developers to help newer developers become newer "good" developers rather than newer crappy developers who produce crappy modules.

6) Drupal the Product, because it includes, includes the people on That is, the Drupal Community is part of Drupal the Product. This ties back to goal #1, but the caveat is that not all developers are equal. "Good" developers are more valuable than crappy developers (for some definition of good and crappy). Some crappy developers can be turned into good developers with more or less guidance, but not all. (Of course, there is no way to point that out without violating the western-Anglo-Enlightenment axiom that everyone is equal in potential, and to say otherwise is creating an "inner circle/elite/cool kids" class and "plebs". That statement is of course utter BS, but we cling to it anyway.)

7) It is also true that, since crappy developers can in many cases be turned into good developers, we want to keep them around long enough for them to turn into good developers rather than scaring them off to GitHub where they will either not improve Drupal the product or not become good developers, or both.

There are also technical constraints to consider, especially the fact that we have a single namespace. There is one, and only one, "filefield" module in all of Drupal the Product. It lives at That is the only and only filefield module. Anyone who doesn't think that and aren't confusing is diluding himself. The former is the "better" namepace, and what someone is likely to look for first. But how many people know the history of those two namespaces? Which really is the "real" module? A "just throw anything anywhere" approach will lay waste to the namespace, a problem we are already seeing. Hence the proposed "no rules" #0 approach is not even worth considering. It is logistically not even possible. is not GitHub, nor does it aim to be. Drupal the product benefits from not having projects forking on a daily basis. We do not want to encourage that, and if someone does routinely fork modules those forks should be considered second-class citizens. Yes, there, I said it. Note that module duplication is not inherently the same as module forks. Those are two subtly different issues.

So given all that, the underlying question here is "what structure will best serve Drupal the Product?" Our goals are: More developers. Higher-quality developers. Higher-quality modules.

Let's try to keep the conversation around those goals, and an understanding of them.

About our goals...

webchick's picture

So given all that, the underlying question here is "what structure will best serve Drupal the Product?" Our goals are: More developers. Higher-quality developers. Higher-quality modules.

Here's where I emphatically disagree.

Have you seen some of my early code? It was complete and utter crap. The first version of Quiz module was an absolute joke, it didn't even contain a UI because the other SoC student responsible for it frigged off and didn't get it done. It had security holes. It mis-used APIs because I didn't understand what a hook was until halfway through the project nor how to write forms using the new Form API syntax. Etc. In fact, I misunderstood CVS so badly back then that I accidentally committed code to a completely separate project. :P~

By the standard of higher-quality developers and higher-quality modules, I should've been denied access to contribute back in 2005. Period. I'd like to think that my contributions since then, however, have made up for my initial stab at Quiz module. :P

Because nowadays, I can do some pretty tricky stuff code-wise, and I know my way around Drupal well enough for Dries to have appointed me a core maintainer. How did that happen? Because I've had 5+ years of immersion in the community, and have been able to put my code out there, have it reviewed by my peers, and learn new tricks from said peers by reading their code. Would I have gotten immersed in the Drupal community if my first interaction had been someone telling me my code was crap? HELL no!

And Quiz module? Quiz module is now an awesome project with Views integration, Actions integration, and tons of cool features. Why? Certainly not because of me. But because others saw the initial stab I'd done had potential, and built off it to make something awesome and amazing. (Which is a great thing, since I've not been able to seriously touch contributed module code in about 3 years now, even though I'm finally a "higher-quality" developer. Stupid Peter principle. ;))

So when you say:

"Is simply increasing the numeric quantity of people who write code that lives on a good thing? If your answer is "Well of course, duh", then you are doing Drupal a disservice."

That strikes me as completely illogical.

We need constant influx of new, passionate people, so that they can become the rockstar developers of the future and can help mentor further new, passionate people who come in after them. People who are already high-quality developers contributing high-quality modules are an extremely rare breed, and tend to either get a) burnt out to a crisp after a year or two, or b) get "promoted" out of coding to non-technical community leadership positions should they be unfortunately blessed with halfway decent communication skills.

Now, responding to some other points....

Because "Drupal the product" includes, we want the quality of the repository in toto to be high.

No. We want to be the central collaboration hub of Drupal development activity. Because that means more eyes on code and higher quality code results.

To address our end users' needs, we want to make it easy to find the high quality modules and identify them as such. And we do that in several ways:

  1. sorts by usage stats, so most used modules are first.
  2. Search results also sort by usage stats. is the first module when you search for "ldap": is like the 8th or 9th result. If ldap end sup taking off on its own merits, it will eventually overtake ldap_integration.
  3. Each project page contains a block with statistics about how active the commiters are, how many bugs the module has, etc. to help gauge a project's health.

Some built-in review step is, really, our only opportunity for more experienced developers to help newer developers become newer "good" developers rather than newer crappy developers who produce crappy modules.

That is totally and unequivocally untrue. My CVS application was totally rubber stamped, because I was in Google Summer of Code. And while I did get reviews on Quiz module's code from my mentors, the "real" place I learned everything I know about Drupal is in the core queue by reviewing patches.

There are so many ways we could do code reviews other than as a blocker to access granting! We could have opt-in code reviews that give people a little badge on their project page. We could have a team who gets a view of new projects/contributors and checks in on them to make sure they're adhering to best practices. We could do "review competitions" at Drupalcons and/or local user groups, where we challenge each other to find and fix the most problems in a module. The possibilities are endless! Let's not box ourselves into thinking that bludgeoning people over the head as soon as they get in the door is the only way to mentor. It's in fact one of the worst ways possible.

Not disagreeing

Crell's picture

Angie, I don't think you're disagreeing with my comment as much as you think. :-)

I did not say that "more developers == good" is untrue. Nor did I say it was true. I said that taking that statement as axiomatically true at face value without questioning why we make that claim is doing Drupal a disservice. A Faith that cannot stand to be challenged is false Faith. That applies to open source just as much as to religion. :-)

How is it illogical to question whether "quantity automatically results in quality" is an inherently true statement? Can you logically demonstrate that statement is true? Because there are stacks of reports both anecdotal and formal that say "one really good developer is worth more than ten mediocre developers." That does not mean mediocre developers cannot become really good, or that non-really-good developers are useless. I am not saying that in the slightest. I am challenging someone to prove the "conventional wisdom" that more people => better, when it's lunacy to claim that all developers are equal, and shear stupidity to pretend that just because we're open source that we're not subject to the same "complexity of scale" as every other grouping of people in the universe.

I also said, quite explicitly, that many many many people who start off as crappy developers can become good developers, and we want that to happen. I cited myself as an example in my next comment, and you cited yourself here as well. I am 100% in agreement that we should make that happen as often as possible.

However, the initial-module-review process is currently the one and only time we structurally have a process for doing so. Every other example you cited is completely ad hoc, happens when it happens, which means it often doesn't or is dependent on who happens to be in #Drupal that night and in a good mood. Clearly, not everyone agrees that structure is inherently a bad thing, judging from this thread. No, we should not have 50 forms to fill out in triplicate to get one line of code added. That's nonsense. But "here's the deep end of the pool, the community will provide(tm)" is demonstrably a crappy answer much of the time. As sun said above, when done properly the more "formal" review step can go well and be very educational. Is it our only avenue to help turn new developers into great developers? No. But it's the only one that we have any actual process around right now. That's why a lot of people latch onto it and want to keep it, in some form. That we right now have lots of ways to make an end run around it is a bug, not a feature, from that point of view.


bonobo's picture


However, the initial-module-review process is currently the one and only time we structurally have a process for doing so.

And people who get cvs access by becoming co-maintainers bypass this review process, which renders this "structure" as a point on a spectrum containing porous and whimsical.

So, given a choice between a faux structure with enormous holes, or a loose structure that provides a cleaner path for increased involvement in the community, I'll take the latter.


IceCreamYou's picture

Webchick, this is exactly what I meant to say earlier, but you said it much better.

When I started with Drupal, my code was absolute crap. Today, I think of myself as an expert Drupal developer. I spent thousands of hours of my free time almost every day over the last 3 years programming Drupal modules because it was fun, and there was no one to say "no, we don't want your code here, it's not high enough quality." Had that happened, I would have gone elsewhere permanently.

When I applied for a CVS account, I didn't have any code -- I just had an idea for a module. And my idea was a duplicate. I sent someone an email saying "I have an idea for a module that does XYZ, can I have a CVS account?" They emailed me back saying "That already exists, do you have any other ideas?" And I said "yeah, I was thinking about ABC," and they said "oh okay, that sounds good." That's not a system I would advocate today, but it's certainly one that got me involved and kept me involved.

I would like to think that moving forward, the Drupal community will continue to support people that know nothing about Drupal or programming for it. I see Drupal as a developer tool, but many of us don't start as developers.

#1 for now

Crell's picture

With all of that said, I am going to put my backing behind #1... with the addition of having better tools for turning crappy developers into good developers and for separating crappy modules from good modules. (And lest anyone think I'm just being an egotistic prick, I would not by any stretch of the imagination say I was a good developer when I first joined Drupal. I was squarely in the crappy category for a while.)

While a human is the best teacher, as others have pointed out we already have too few people handling applications now, and the security team is grossly overworked. We simply don't have the resources to review every new module. We barely have the resources to review every new contributor.

Others have mentioned ideas in this thread for how to better automate module data so that we can tell if it's good or crappy, by some definitions of good and crappy, so I won't repeat them.

I will shoot down the proposal to only review "stable" releases. As others have noted we already have a critical problem with module developers "avoiding responsibility" by staying in beta forever. (And yes, I have already said publicly and will restate that I consider that to be irresponsible.) Let's not give people even more reason to avoid responsibility.

The other factor is that with sandboxes being more useful than they are in CVS now, and with, eventually, every issue getting its own repository, the dynamics of what constitutes a "fork" are going to change radically. We don't fully understand it yet. We may find that the situation gets worse and #2 becomes a dire need in the future, or we may find that a more robust set of tools on d.o solves those problems for us without us even realizing it. We simply don't know.

I would rather hold off until we do know, and then revisit this question in the latter half of 2011 once we get a sense for the new landscape. We should be open to revisiting it, though. This is not the end of the conversation.

Wrong problem

Acubed's picture

The whole issue of using Git is being seriously over-engineered. Git is a decentralized revision control system, we've all heard that, but it's such a hugely important point. It's why Git was written in the first place. If you haven't seen Linus' talk on Git, go see it at YouTube: . Even if you have seen it, see it again, and pay attention to what he says about network of trust, commit access, patches and tarballs, branching and merging. In particular: "So in practice... there will be a thousand or maybe twenty thousand different branches, but in practice you won't ever see them, because you won't care. You will see a few main branches, maybe you will see only one--In the case of the kernel, they really only look at my branch, so even though there are lots of branches, you can ignore them. The way merging is done is the way real security is done, by a network of trust."

There is no canonical repository with Git. With Git, Linus is no longer "The" Linux kernel (short of legally owning the trademark, but with respect to the actual code). This fact is being entirely lost by people who still want to think hosting is keeping the same centralized model with commit access. But it doesn't exist. It is easy for anyone to commit code now (without resorting to keeping a backlog of patches), and the only discussion that is relevant is "What subset of code will voluntarily host, out of all the code being stored in Git repositories?" is not the authoritative source for what modules may be published, it is just one place out of many to get hosting (it might be the only place to get branding, etc, of course). There's no reason that every single project hosted here couldn't up and move to or right now and continue development in a free environment with no significant regulations except that it's posted under an open-source compatible license. A more likely solution is that projects will start at GitHub (since it's cheap to start a repository there), then go through the (less painful nowadays) process of creating a project, and then once it meets all the criteria for hosting, it comes over here. Is that what we want? Or is a place where anyone can publish their code, publish their branches, and easily collaborate?

Having thousands of branches (each repository has its own branch, remember, unlike CVS where there the branches only exist on the server) is the reason why Git is so popular, and makes collaboration easier. Git is not an excuse to play by yourself in your own sandbox, it's a reason to collaborate easier with other people. That's even the slogan of "Social Coding". If the site is looking to be successful, it needs to at least emulate those features that made it successful: branching, merging, social coding. That is: As few regulations as possible for hosting code, since commits are no longer scarce economic resources, and possibly higher standards for being assigned a module name which is in limited supply. Note how GitHub gets around this, by identifying repositories with the username... it's doubtful could use this because it's assigning more than the project name, it's assigning a globally unique module name. And there's the answer:

Don't focus on Git, Git is the wrong problem. Focus on where there are significant economic resources: Naming modules. That's what does best, not hosting code, if we want to determine who gets a module name, focus on that problem instead. Otherwise there is no excuse for restricting code if the intention is to create a vibrant community. (If Drupal really wants to solve the economic problems associated with naming, it should move to using URIs for naming modules, but that's a discussion for another target audience.)

All correct, all abstract,

sdboyer's picture

All correct, all abstract, and all unhelpful.

  • The fact that git itself has no canonical repo doesn't mean that there are no canonical Drupal repos. We've got an update system that every non-hacked version of core ships with, which has the effect of making a repo canonical.
  • The namespace problem is real because we chose to stick with Project*, and that imposes certain restrictions. Like namespaces. Even without Project*, though, there's still namespace issues because of how PHP registers function names and the de facto conventions around project/module/function naming.
  • And most importantly, "Git" isn't what we're discussing here. We're discussing "Drupal's migration to git," which has many particulars layered on top of that which is entailed by git itself. The concerns being discussed in this issue necessarily go hand-in-hand with the migration, even if they don't go hand-in-hand with git itself.

I can understand how it might

Acubed's picture

I can understand how it might not be helpful, at best I didn't (directly) endorse one of the options, at distant worst I just derailed a thread with over a hundred comments. The primary intention was to take on the belief that somehow Git would stop collaboration, and that there is no reason at all who gets to commit what should be at issue (aside from keeping the content on-topic), that by itself. You can't think of the implications that will have on the question posed in the thread?

If Drupal continues to contact only servers for getting updates, that's a deliberate (if unnecessary) design choice, there's no reason it couldn't contact and grab new tags, for instance. And concerning the migration in the context of the thread, it means that the decision of who gets to create Git repositories is irrelevant, it's about who gets to create projects (that is, who gets to name the modules). Users should already by able to create and fork repositories, without needing the global name.

If you really want to see what I would do about it, here's my criteria, in bullet points (because we like bullet points):

  • All users receive public-only hosting for Git repositories and all the social tools built around it, that satisfy the topical criteria (Drupal module, open source license, etc), given under their username and then a name of their choice.
    • webchick suggests using the node ID, this would be structured more like Github's Gists, and while maybe that's what is wanted in some cases, it does not take into account the multiple branches that are created when multiple users work on a project.
    • Alternatively users can link to or mirror external repositories.
  • Remove dependencies on a particular naming style, including Git branches (this is happening), project names (not too big in the context of the migration) and/or module names (this would require a major revision increment to Drupal core at best).
    • Come on, we're engineers, we can figure out how to structure function calls if the naming scheme ever changed. Naming based on the hash of the module name, some other name munging, or using those shiny new function closures spring to mind.
  • Community review for projects looking to get a name in the global namespace (outside of user namespaces), possibly with a minimum time to approval.
    • This would encourage users to fork+merge existing projects to add functionality instead of creating new projects.
    • This would not likely be the correct solution for a small, growing community, the rules would need to be react differently to the conditions to grow to this point, but I can dispose with this because the community has matured enough to skip to this point.
  • " hosted" (or whatever) branding could be revoked if the official project repository (the approved one kept with the project) does not meet quality control standards, but the code does not go away (short of violating the terms of use agreement).

Sorry, I was feeling

sdboyer's picture

Sorry, I was feeling especially snarky when writing my original response. My choosing to say "unhelpful" really has more to do with the fact that some of your point, at least, seems to be taking the discussion here in isolation of various other plans that have already been laid for the migration. And to be fair, I'm the only person who probably knows ALL of those other plans. But yes, it is somewhat tangential to the issue at hand. However...

The primary intention was to take on the belief that somehow Git would stop collaboration, and that there is no reason at all who gets to commit what should be at issue (aside from keeping the content on-topic), that by itself. You can't think of the implications that will have on the question posed in the thread?

I believe that git will make collaboration easier. No question. The concern, as I understand it, is more of a (not necessarily illegitimate) fear of change to how our collaboration works now. Over the years, Drupal has been wildly successful at stitching together CVS, issues queues, and patches into a collaborative environment with wide participation. IIRC, there's some statistic somewhere that says Drupal (core) has the largest number of contributors per line of code of any open source project, ever. Git introduces the potential for people to collaborate in ways that are less public, less inclusive, less accessible to folks just coming in, BECAUSE so much of it can happen directly in the vcs, behind the scenes.

The second half of that quote is a little hard to follow, so I'm assuming you're arguing that "who commits" shouldn't be relevant, and you're wondering if I see implications for the model we adopt here. If "who commits" refers to who publishes/pushes - well that's just silly. It'll always be relevant. No project of any considerable size would last long without integrators who tie it all together. If "who commits" refers to who actually makes the commits, then I agree entirely - integrators should be able to merge commits from wherever they want, d.o or not. Over the course of the migration, I've invested a LOT of effort in ensuring that the d.o git environment is as not-d.o-specific as possible. And I'm pretty happy with where it's ended up - basically the only restrictions will be that project & sandbox repos are ff-only (that is, receive.denyNonFastForwards will be set on those repos), and there'll be limits on their size to prevent spam. Implications for this process? I think I've already thought this particular piece through pretty well, so I don't see any. Once we introduce per-issue repositories, we'll be in great shape.

If Drupal continues to contact only servers for getting updates, that's a deliberate (if unnecessary) design choice, there's no reason it couldn't contact and grab new tags, for instance.

You really ought to spend more time with the update system. It currently operates using metadata attached to info files by the d.o packaging system - it does not use, nor does it care about, any underlying vcs. There's the CVS Deploy and Git Deploy modules which enable vcs-based update management. The other piece of it, then, is the release XML - good luck getting github to generate that. Features are the only other established update system, and they are distributed, but they're also completely separate from the update system. So, sure, if you want to spend the next six months, maybe a year, creating a whole new update system for us, then maybe. But this is what really makes your argument tangential - strong statements with little regard for our current reality.

it means that the decision of who gets to create Git repositories is irrelevant, it's about who gets to create projects (that is, who gets to name the modules)

One project == one repo. Add a project, a new git repo pops out. Delete the project, the repo is blown away. The two move in lockstep. I'm not even sure what to say "who gets to name the modules" - module naming is tied to project name purely by convention. Nothing in the code requires the container (the project) to have a name anything like that of the contained modules.

Users should already by able to create and fork repositories, without needing the global name.

Just pulled this out as an example of sort of reasoning you're doing that I'm consistently having problems with - the fact that doing something (cloning) in git is easy doesn't make it self-justifying that an entire web-based interaction architecture built overtop of git will necessarily support all the same features & workflows.

All users receive public-only hosting for Git repositories and all the social tools built around it, that satisfy the topical criteria (Drupal module, open source license, etc), given under their username and then a name of their choice.

Not a new idea. See . Eventually I'd like sandboxes to function like this, but it wasn't feasible to do in phase 2. We had to go the middle route.

Remove dependencies on a particular naming style, including Git branches (this is happening), project names (not too big in the context of the migration) and/or module names (this would require a major revision increment to Drupal core at best).

Yep, we're killing branch/tag naming restrictions, though there will still be restrictions on the branches/tags that can be pulled up for releases. There are no conventions on project names, so I'm not sure what that's about. As for changing the module naming conventions...well, if you could convince people to do it and provide the docs, and migrate all the code? Great. But it won't fly unless it's as simple and intuitive as what we have now - prefix all function names with the module name. Most importantly, though, it's completely outside the scope of the git migration. For all we care (with small, not-relevant-here exceptions), all 8500+ project repos could be filled with binary data.

Community review for projects looking to get a name in the global namespace (outside of user namespaces), possibly with a minimum time to approval.

In other words, #2.


sun's picture

@Acubed, I enjoyed reading your comment. Would like to see more from you. Excellent thinking.

Of course, @sdboyer has some legit and real counter-arguments. But that doesn't destroy your very valuable input.


Daniel F. Kudwien
unleashed mind

+1 for #1

bonobo's picture

I will admit that I came into this thread favoring option 1.

Having spent a chunk of time reading through the comments in this post (and talking about them here in the office), #1 is still the best choice.

Would it be great to have code reviews on projects? Yes, absolutely. Would it be great if we could provide every contributor with mentorship/support to write better code? Yes, absolutely.

But these are things we have been talking about for years, and with the exception of some specific contexts that are designed to foster that type of support (Summer of Code, GHOP, some code sprints,, etc) these supports have not materialized in any systematic way.

More contributors doesn't necessarily translate into better code. But more obstacles will definitely translate into talented new contributors putting code elsewhere, and/or not buying into the community.

Talented people have options; we forget that at the peril of bringing new blood into the project.

So, +1 for Option 1. If we get the bodies/budget/structures to really support additional review, we can implement them at a later date.

kyle_matthews wrote: In any

Dustin Currie's picture

kyle_matthews wrote:

In any case, there always has been duplicate/redundant/bad/stupid/insecure/whatever code and there always will be. So if people have trouble finding good modules I vote that we develop better filters not stop projects that someone doesn't like from being created.

I want to highlight this, because it gives real insight into a currently existing problem that will not be solved by either of the two initially proposed options and indicates a solution.

Yet another idea to discuss

Dustin Currie's picture

Let me throw another alterntive idea out there:

What if we let every user create projects and releases?

The default project page could be located at:[uid]/project/projectname or[uid]_projectname

Packaged releases and project pages could contain warnings saying this is not a community reviewed/approved project.

When a threshold is met, the project could be escalated to

Anyone and everyone can not only commit code, but can also create projects and releases. Those projects and releases could be used with drush, makefiles and distributions. No approval would be required. The goal of welcoming more contributors and making it easier to jump in would be greatly improved upon.

Potential negatives: The amount of garbage code would explode. Exponentially. The possibility for malware hosted on d.o. would exist.

Yet, by allowing only vetted projects (and not just projects by one-time vetted users) to be located at project/projectname the stated goal of making it easier to find quality projects could be further improved upon.

Potential negatives: It would be difficult to build consensus on rules for moving a project from a personal experimental project to a community approved project.

There's probably a lot more issues that I'm not seeing. This would be a radical change.


sdboyer's picture

See for a lot of related discussion. What you're describing is another possible model, but one we ultimately didn't go with in the attempt to keep the code requirements a little lower. In retrospect I kinda woulda liked to pursue using user-namespaced sandboxes with shortnames a bit more, but it's water under the bridge now (for phase 2, anyway).

#1 (with eventual modifications)

arianek's picture

After arguing more on IRC last night, sleeping on it, and reading all of the new comments here, I'd like to withdraw my support for #2 as a fallback. Mainly because I don't believe right now that there will ever be enough reviewers to make this not be a hurdle/bottleneck that discourages potential contributors.

For this initial completion of the migration, I still support an unmodified #1.

In the future, if there were the resources for it (ie. a LOT of new volunteers and/or automation) I would support a #4-esque workflow. And I would also enthusiastically support the feature improvements noted in webchick's comment ( - Reviews/ratings, badges, a feed of sandbox projects, and opt-in detailed code reviews) as a means of helping both devs and end users find what they need and keep the quality of formal releases high.

Identifying duplicate / similar modules

DamienMcKenna's picture

How about not discouraging duplicate / similar modules but providing improved methods for a) developers & site maintainers (infrastructure group I guess?) to tag related modules, b) visitors to see related modules to better find the one that suits their needs. Maybe something as simple as either an extra vocabulary or even just expanding the Module Categories list to add hierarchies would be sufficient?

A key part, though, would be finding a good way of displaying this information on the project page itself - maybe replace the current "Related Modules" list with a) a list of "Complimentary modules" (e.g. XMLSitemap and Nodewords are complimentary) b) a list of "Similar modules" (e.g. Nodewords, Metatags and Integrated Metatags are similar/duplicates)?

But this could be functionality for after the initial migration.

Option #1

infojunkie's picture

...for many reasons:
* Decouple infrastructure from process
* Don't introduce too much change at once
* Don't add barriers to entry

I consider open source development a bit like the process of evolution in the natural world: any number of new strands (modules) can be introduced, and some are more successful than others. But I don't think it's possible to design a process that "enhances" evolution, at least not with our current understanding of the forces driving it. One thing's for sure, the redundancy of having multiple modules doing similar functions is essential for better modules to emerge. So let's not tamper with that!

On the other hand, we can help site builders better navigate this ecosystem. When I search through existing modules for a specific piece of functionality, I always feel I am stumbling on the remains of 5.x or abandoned modules. That's a usability problem. I support Damien McKenna's idea of related modules; in fact, the "complementary" relationship can easily be determined by the dependency entries in the module's info file. Exposing this list for search and display on D.o. would be a great start.

Option #3

SepehrLajevardi's picture

Being realistic the community cannot afford such amount of time to review every project creation. So as a result we will get tens of quality, fully functional projects distributed on various code hostings but yet unpublished on D.o. I myself had an application in the queue for about 3 months and I got no review. So I think faster fails in this case. Also why should we spend such amount of time to review a project authored by a community highlighted member just like the ones webchick mentioned.

In the other hand, there are tens of orphan projects on d.o with no initial commit, there are lots of non-standard ones on CVS due to the creator's lack of commitment to the process. So I disagree to continue to use the current method.

Also this might be a good idea to make the initial review done by running the code through the coder module automatically.

Wrapping these all, I vote for dww's third option.

#1 with an idea about approval & review guidelines

chachasikes's picture

I'm into the idea of #1, so as to get things going and see what we really do with our new sandboxes.

Also, regarding making the approval process as transparent, easy-to-understand & with lower barriers, I had some thoughts:

  • The Mozilla/peer 2 peer university/school of webcraft folks are talking about badges & credits for learning 'webcraft.'
  • At the Mozilla open learning conference, we were talking about peer-to-peer assessment & badges like 'I know CSS 2.0'. (I told them about our peer-review process for patches/CVS accounts.)
  • They have made a map of all web skills generally involved in being a web developer (the map has HTML, CSS, PHP, server admin, etc.) - naturally, I wonder what this would look like for Drupal developers who are working on becoming better developers.
  • So... for newer developers & reviewers, it might be helpful to have some sort of checklist to compare which criteria their module, theme, patch, product, etc needs to meet. I definitely used the CVS application guidelines to understand what was expected of me as a good developer.
  • I took the CVS application guidelines very seriously. I didn't want to 'hack the approval process' by just becoming a co-maintainer. Validation is pretty important for me, even if it is just a checklist that I read myself. As a learned, I wanted to be able to make sure that I had really covered everything. The need to clarify what is expected of a 'good developer/contributor' isn't going to go away no matter where in the process we might put up a review wall. And I would bet that there are many ways of being a good developer. Everyone is different & no one is perfect - so probably there would be a set of 'positive developer attributes' - some necessary, some optional based on changing circumstances. Some people are going to write beautiful code, some people are going to be very creative, some are going to finally solve problems everyone was having. So many of us look at all the many collective contributions and learn a little from all of them.
  • Things that might be on a checklist: My module isn't redundant. I reached out to the community to look at my experimental project to turning it into a real project. I can manage an issue queue. I asked for a security review. I'm ready to make module releases... (There are many more of these.)

Something like this could make the full process visible for learner developers, while giving a really clear sense of where a particular module/theme is at for better filtering. Sort of like a "module readiness gradient."

+1 on the filtering & bubbling up of good modules.


chachasikes's picture

crazy wifi submitting my comment three times - sorry. I don't have permissions to delete the duplicates...

to every developer (most of us) against review... for us

mlncn's picture

What is your plan for keeping new contributors from being 7th-class citizens, as they are now?

Please see Our CVS account application requirements are obtuse and discourage contributions and Determine/finalize technical requirements for post-Git migration project approval process (both marked as fixed) and tell me how to prevent us from continuing with the current situation.

About the current situation: I've been trying to pick off a few people from the waiting seven weeks end of the application queue. (If someone with super cow powers could just short-circuit this one that would be great. And someone care to argue for this unfortunate contributor?)

Seriously. If any version of the process we can come up with for reviewing contributions is utterly unacceptable to you personally, you should be first in line for knocking barriers as flat as possible and probably handing our new contributors lemonade.

Proposal for reviewing code contributions: a new project (for new contributors, or, in my vote, for anyone) needs someone who has had three projects approved to press a button to approve it. No special reviewer role. Just a privilege to anyone who has had three projects of their own approved. We would still have guidelines, but a much more open process and no bottlenecks. Officially an issue should be created but there will be no enforcement of that, just the public record of who pressed the button.

Caution: My mind will explode if people who say this is too onerous to apply to them argue that it makes it too easy for new contributors to get projects approved.

benjamin, agaric

option 3 open sandbox is great

sime's picture

I like Option #3. As long as contributors get access to the package system for their sandbox projects. This gives their projects a good chance of being tried in the wild.

If I was to suggest an option it would be along these lines:

  • The account review process should die.
  • Everyone can commit to their own space. Everyone can create projects. Simple "I agree to GPL" checkbox, etc.
  • People can define whatever namespace they want in their packaged version. If I think I've got a better "views" module then let the ecosystem decide that. Certainly the tools can warn me, which is great, because it helps me find the existing Views module.
  • Focus on tools of discovery of the best modules. Eg. give users the chance to display a "favourite modules" list and use that data for discovery tools.
  • Namespacing problems in git are an artificial problem, it shouldn't matter how the data is stored.
  • Namespacing problems in clean urls are an artificial problem. Just add the username to the url like github.

+1 for #1

dragonwize's picture

I think #2 is a bad idea on any day. There would be to many road blocks, devs would just leave their code unapproved.

Is there issues with #1? Sure, but it is how we do it today and so the community knows it. So for lack of a better option that we would be able to do during the migration and would not completely throw the community for a loop, option #1 is best, for now.

The ideas of having code sandboxed to the user until they are ready for a project sounds nice but it has many problems too. It needs to be fleshed out more to make it perfect. Is there a perfect way? Maybe not but there is not a big reason I see at this time to move from one way to another when they both have problems. All we end up doing by continually changing things is using the community as a test subject, and while that is fine on a very small scale, it has a horrific effect when done over and over again. For an example of this, look at facebook. They keep changing the interface, to their credit, to make it better. The problem is not the fact that they do not make it better in some way. The problem is that they keep testing their test designs over and over and over on their community at large instead of fleshing out a better way and using the entire community only when they are sure they have an idea that is worth trying. Because as open as this conversation is, the entire community is not participating, and their is not a single idea I have read that does not have as serious of issues as our current process.

Whatever the choice don't make the community at large go through a continually changing system as we test every possible method on them. It is just plain discouraging.

why not a poll to vote for the options

vwX's picture

Just saying.

Because discussions are much more meaningful...

webchick's picture

Polls don't capture feelings/thoughts behind the votes, which discussions like this do. I don't think raw numbers is the way to make this decision, I think it's carefully weighing pros and cons, spotting trends, and making sound arguments for/against other options.

This discussion basically strikes at the heart of our contributor community, and whether we ultimately want it to follow more of a 'bazaar' model, or more of a 'cathedral' model. It's not something I'm comfortable leaving to poll module to decide. :)

Option #1

btopro's picture

I see android market vs. Apple market. Sure there's less "crap" in the Apple app store then the Android store (also debatable statement I know) but the Android store "feels" more inclusive and open. I see less people complain about the process that google has in-place with android then the overlord approach of Apple. Yes, you'll get bumper crop, modules that "don't do things the right way", poorly coded, but you'll also have more eyes / voices on the big projects because people feel more included in the community.

I'd also hate to see winners and losers picked by an approval process. Yes there would probably be more high-quality modules (and less of them) but I think the usage statistics, committer track record, and maintenance of the project page often speak volumes. What about a prominent "What to look for in a maintained module" page / video on the module search page. Give some rules-of-thumb, show what they look like and let usage statistics drive winners and losers

Quality, not Approval

zzolo's picture

Sparked by @btopro comment but applies to lots of comments above:

Just a quick note from someone that does application review currently. It is a very rare occasion that someone does not get "approved". Outside of the obvious module duplication, the review is just ensuring a basic level of quality and community knowledge so that it is more beneficial to the community going in. It is not a matter of saying "you can't come in, your code is not good enough", it goes "your code could use some work before we let you have free range of our repository" and then it is an iterative process to help the person become a better contributor to Drupal.

So, when referring to the application process, it is not a secret handshake, but yes, it is a hoop to jump through (which I personally find beneficial). Also note that all application comments are done in the open in the queue, and that there is lots of documentation (including subpages) on what reviewers will probably look for and the how the process should happen.

Code review and the application process

bonobo's picture

The need for more code review (which really is inseparable from the need for more reviewers) has been around for as long as I've been a part of the community.

Reviewing code falls into an odd middle ground between writing documentation, writing code, working on the handbook, testing and writing patches, etc. Because of this, (IMO) it often gets overlooked.

As a step toward making code review more of a "thing" within the community, and making code review something that happens for modules that have already been released within contrib, I put together a wiki page that lays out some potential steps for doing code reviews.

I'd love to see more visibility for code reviews, in general. Providing reviews of modules in the contrib space is a great training/learning tool for new developers.

So, feel free to edit away on the wiki page.

And if you think this is a crap idea, feel free to say that too!

Thinking more and more about

pivica's picture

Thinking more and more about this thread, except that I am still voting for #1, I am not even sure any more why module duplication would be considered by default like a bad thing?

Except a case when somebody literally copy&pasted someone else module, and changed couple of text string - this is something that is definitely a bad thing and need to be sanctioned I can think more then one use case when module duplications is a valid approach:
- For example there is a module for that but I really don't like how it function under the hood. I think that my solution is better and more scalable,
- Yes there is a module for that, but it is not well maintained, and messy, so trying to fix that and add additional stuff that I wont is just not an option for me it I would need to spend too much time on it,
- Ok there is a module for that but module maintainer and me are very busy. What I wont to do will need lot of discussions, patches, reviews.... I just don't have enough free time for all that and I want to invest that time/energy in even more development for my new module and add additional functions,
- There is a module for that but what I wont and what module maintainer wants are different things - our future plans just goes into the different directions...,
- And yes there is a module for that, I could even create patch for it which will probably go into the next release, BUT I wont to create my module with my solution and go with my way.

Typical case is image gallery solution for D6 - there are tons of options and modules for this and what's wrong with that? Maybe it would be better that instead 10 modules and 10 authors, all 10 authors sit together, plan and creat 1 great image gallery solution - but if they don't want to do that what is the problem with that?

Sure no solution is perfect one, and every solution has problems, even #1, but I do not know why should we change current contrib space organisation. I think that developers need to have rights to decide what they want to do and how, and community needs to have tools to review/evaluate/share contributed modules infos. In short let the coder create module how they want and let community decide which modules they want to use - good/well maintained and popular modules will get broader attention and will probably grow, bad/not well maintained modules will probably die after some time.

This do not means that I am advertising module duplication - I think that each module developer should first evaluate others solution if they exist and then decide what to do - create patch for some other module or write new module and publish it on

So I am more for better review tools - so end users can more easily decide which module to pick - and redesign of already done a lot of nice stuff in this area - sorting modules by popularity (most installed) by default for example.

Here are two additional review information that would maybe help in module review process:

  1. One of things that I would like to see for module reviewing is some kind of security information on project page - is security team reviewed this module/theme, is it passed review and when was the last security review from security team.

  2. Also maybe some kind of 'module/theme value index' - this can be some auto-calculated number from 0 to up (maybe without max limit) and we can use couple of factors for it calculation like:

- number of reported installs divided with some coefficient (maybe with current contrib module count)
- if module author has more modules, add somehow 'module value indexes' from other modules
- module which author is active and longer on gets additional 'module value index' points
- If module is more actively maintained 'module value index' is bigger
- maybe if module passed all coder checks add more value index to it

Something like that - nothing too complicated (others users already had simmilar ideas in this thread), just some number that can help some user to really just download some module and continue to work something else - so if he has couple of options to choose but he is really just end user and don't want to invest his time in additional research then give him a nice big 'module value index' so he can pick better module.


Drave Robber's picture

I believe that #1 or #2 doesn't really matter as the sandbox system itself will be the game-changer.


I'm one of the last people with 6-digit user IDs. :P

In ~3 months with Drupal, I've written three modules (pretty tiny though – some 10kB in total), and another one is in the works. These modules are used on a production site and pass Coder review. I've not bothered to apply for CVS account, mostly because my modules are specific enhancements to the core poll which (I reckon) would be of interest probably to two or three sites in the whole world.

When sandboxes are introduced, I'm going to put most of my stuff out there; I'll consider any comments or suggestions that might (or might not) come in, but I won't care about projects being 'official' or not. Perhaps I'd even prefer to keep them in the sandbox.

Somehow I suspect I'm not alone in being only happy to share but not giving a damn about status.

My vote

ximo's picture

The most straight forward migration path (#1 from what I understand?) for now.
Then continue this discussion based on a new excellent webchick summary™ :)

Too many modules?

KarenS's picture

Lots of issues here, but the main one circles around the idea that there are 'too many' modules. Even duplication is only really a problem when it is combined with the sheer volume of modules.

Volume is a problem for developers because it's hard to even figure out which module might solve your problem before you create (yet another) solution. Volume is a problem for users because it's hard to separate the wheat from the chaff.

But I would argue that diversity is good. Even if there are three modules to solve a problem, the first one may or may not solve it well (or well enough for your use). And it's not always possible, or even desirable to create gigantic modules that incorporate every single feature that any other developer wanted to add. Some maintainers are open to incorporating your extra code, but in many cases it's difficult or impossible to do so without creating some Frankenstein monster that is hard to maintain and really doesn't make anyone happy. It can be done and it has been done, but it's not automatically 'better' to pour everything related to one topic into a single module. And from the user perspective, I often want to find the simplest module that solves my problem, not necessarily the one with every possible bell and whistle.

The assumption is that the way to reduce volume is to put up as many obstacles as possible to keep modules from being created. And that creates another problem -- finding people with the interest and energy to review the applications and determine what is 'good' and what is not. In reality, you won't review a module until you need it, and you review it by trying it out and finding out its strengths and weaknesses. And most modules aren't well-used until they have releases, not before that. Once released we have a huge base of people who might find problems in them -- not just the security team.

So how could we solve both problems? By keeping the freedom to create modules and projects but doing a better job of filtering them and cleaning out the ones that aren't working or aren't well maintained. Some of this is already implemented -- we now have a list of 'most downloaded' modules and maintenance and download stats on the project pages. We could use a richer taxonomy for filtering them, and a better job by maintainers of tagging them appropriately.

And maybe we need a better system for 'demoting' modules. If we added some easy ways for the general public to report that a module is a duplicate of a better-written module or that it is poorly written or poorly maintained, or, obviously, that it has security issues, then the security/maintenance team would only need to follow up on those reports rather than review every project. Maybe in our new system we can 'demote' projects back to sandboxes -- no code is lost, they are still there, but less visible.

Curate the code, not the contributors

laura s's picture

I feel strongly that #1 is way preferable to #2, which strikes me as a lot of work that is going to drive people away, irritate many who stay, and generally give contributing a sour taste. I think it would be a mistake to create a tiered system. We're already at risk of that as the community grows so quickly.

There are many ways to help people cull through all the variety. I don't see it as a positive to prevent someone from contributing something they've done (for a client or employer, perhaps) because they could have spent untold extra hours trying to collaborate with the maintainer of another similar project that does not have identical goals or requirements.

(Time to mix some metaphors.) Forking is a feature. Trying to gatekeep contributors or contributions too much is working against the tide. If we try to freeze the community in amber, the energy will just go elsewhere.

As for parsing through so many modules, we're already making some progress in displaying usage right there on the project page, last commits, open issues, etc are all there to be seen. Crowdsourcing links to similar modules could help (because Solr does miss a lot, depending upon what keywords are in the description). Module ratings like could help. With tens of thousands of active members and hundreds of thousands of authenticated users on d.o, it would be awfully hard to game the system.

Crazy idea: display forked versions on the same project page. Forking developer describes why. Example: "This fork fixes some issues relating to Views integration." "This fork adds code to use Tokens." If a new fork gains more momentum, perhaps it becomes the main project and the previously main project is displayed as an ancestor or something. Module updates stick to their own forks, but site maintainers can see that other options exist aside from trying a dozen patches buried in three dozen issues to try to get an unmaintained project working. should be facilitating collaboration. If it starts (continues?) shutting its doors on contributors early in their projects, they will just go elsewhere. GitHub is already a new resource for modules because of developers who simply hate CVS that much. I'd hate to see d.o policies driving more potential collaborators there.

Laura Scott
PINGV | Strategy • Design • Drupal Development


kiamlaluno's picture

I am for option #1.

Any option where all users who have a CVS account would not be able to create full projects is unrealistic.
What would happen if a project is necessary for, or any sub-domain? If it is necessary to have N sandboxes project to create a full project, then nobody would be able to create a project for, where a project is sometimes created only to have a project issue (the webmasters, CVS applications, infrastructure, Git Migration Community Testing, Project applications, Redesign, Test driven development infrastructure, and Documentation projects).

Kiam la luno renkontas la sunon

my vote: #1

nevergone's picture

My vote: #1, because joining will be easier. :)

Sorry, too late. We are going

Josh The Geek's picture

Sorry, too late. We are going with using sandbox projects.

I go by Josh The Geek.

#2/#3 needs to happen.

phyadmin's picture

If Drupal ever wants to be taken seriously as a robustly engineered "content management framework", then it has to be either option 2 or "3". It cannot be option 0 or 1. Anarchy breeds mediocrity. Not Invented Here causes massive Reinvention of the Wheel.

The module list is a bloated cesspool of half-finished projects and "maybe this will finally solve why my webform doesn't ___." If it's just allowed to grow indiscriminately, it's like encouraging great aunt Linda to just keep eating after she's gone from 140 to 522 lbs. It's like saying "not only do we need more oil, we need it from everywhere!" It's like adding that "one extra feature that shouldn't harm nothin" and never saying "No," with finite development time.

It's simply unsustainable. It impedes the Drupal project, and frustrates new users.

But that's not all.

Clearly, modules have their own eco systems. Mega modules like CCK (now somewhat gone), Views, Token, Rules, Actions, etc all have an abundance of sub-modules that are developed around integrating their features. These types of hierarchically related modules need to be put close together. They should not all show up on a flat list of "10K modules" on the home page. If I have to swim through Yet Another Event Scheduling module, I'll say forget it and write the thing in Grails with a web interface for Drupal to talk to.

Modules with competing functionality need to be identified, and every module should have a user generated rating. From the upvotes and downvotes of users, modules should go through one of the categorization processes similar to modules that ascend to the God category should be integrated into core in the next major release, modules that stay in Fail for longer than a month should be culled.

Just more of the same will not help Drupal evolve.

This discussion has been dead

sdboyer's picture

This discussion has been dead for quite a while. Now that Git's launched, it's been separated out into a number of smaller discussions, but no one is really still watching here.

In fact, we probably ought to get this topic locked.

ok, locked

pwolanin's picture

ok, locked Git Team

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds:

Hot content this week