Discussion regarding Module Duplication in the Code Review Process (and the 'Devil's Advocate' viewpoint).

In a perfect world, the module duplication check would occur before the contributor has written a single line of code. This allows the opportunity to redirect a contributor towards a collaborative effort (building on an existing module) before they have invested any time or effort into creating their own solution. This helps to eliminate wasted effort ... and more importantly, takes place before the applicant develops a strong 'emotional' attachment to their own code/contribution/approach.

Of course, this ideal situation does not fit within the Code Review process, which requires a 'completed' module/theme before an applicant can even submit a full project application. In the code review case, the recommendation becomes that a duplicate module search should be the first thing that a reviewer looks at before considering an application. However, I would argue that even then is too late ... once the code is written, the emotional attachment to the code exists, and any rejection of the contribution is going induce at least some sense of 'letdown' (and thus serves as a de-motivator for the applicant, instead of a motivator supporting the applicant's growth into a long-term contributor). Rule #1 should always be: Don't kill a new contributor's spirit before they even get a foot in the door!

Now, I'm not suggesting that a duplicate module check should not be part of the code review process ... only that a 'proper' long-term solution would address duplication long before an applicant reached the 'code review' stage. As such, the Code Review group should avoid taking on sole accountability for the 'duplicate module' check ... instead, the solution we need to develop should focus on preventing duplicate modules from ever getting into the Project Applications queue in the first place; and then if one does get through, minimizes the letdown when that application is rejected.

Proposed Solutions:
Much of the code review discussion over the last few weeks has revolved around the formalization of a 'Gate' structure for Code Reviews, which, while possibly diverging from the desire to provide 'end-to-end reviewer/mentorship' to an applicant, may enable less experienced reviewers to chip in on some of the more 'standard' chunks of the review process (such as running through Coder, checking for 3rd party licenses, etc). The first 'Gate' in any such process should be the 'duplicate module check'; ensuring that the applicant does not put any additional effort into addressing other gates if the end module is unlikely to be approved ... thus reducing (as much as possible) the sense of frustration associated with running through the approval process only to be denied at the last stage.

But, as mentioned, this only addresses modules which reach the Code Review queue ... it does not help prevent them from reaching the queue in the first place. I believe a good first step towards this goal would be to emphasise the desire to reduce duplicate modules at a number of other entry points into the 'Drupal development' process. To this end, I'd suggest generating a simple 'Reduce Module Duplication' plea to the following pages ... or, if the motivation can not be expressed in a concise enough statement, then create a dedicated 'Reduce Module Duplication' article and link to it on each of the following:

- Developing for Drupal - http://drupal.org/documentation/develop
- Module Developer's Guide - http://drupal.org/developing/modules
- Applying for permission to create full projects - http://drupal.org/node/1011698
- Application review process for Full Projects (what to expect) - http://drupal.org/node/539608
- Creating a new full project - http://drupal.org/node/1068942
- Best Practices for creating and maintaining projects - http://drupal.org/node/7765
- Join forces with others and co-maintaining projects - http://drupal.org/node/23789

A good starting point for the text for such a plea is dman's 'Open Letter - A Proposal to Consolidate' from the 'Similar Module Review' group on g.d.o.

Of course, there will aways be cases where an applicant does their research, determines that their project doesn't significantly duplicate the functionality of any existing modules on Drupal.org, and goes away to happily start coding ... only to return months later to discover that, while they were building their project, another user has since contributed 'Module X', which duplicates 80% of the functionality of their own project. (Or, even worse, imagine that a 'pre-approved' user has contributed 'Module X' while the applicant's own version sat idle in the Code Review backlog queue!)

According to the current implementation, that applicant is out of luck ... even if their module meets all the other code review criteria, makes more efficient utilization of the Drupal APIs, provides a more flexible roadmap for future features, and is generally just a better-coded project in general ... their contribution/code is blocked by the 'Module Duplication' rule, and they are sent away with a suggestion to go collaborate on the 'Module X' project instead. shelleyp summed this up nicely in one of her posts:

More importantly, "duplicate" module developers may not be aware of an existing module, and are only attempting to share what they've created, not necessarily to "duplicate". What you're doing is basically slapping them with a folded up newspaper, which I'm sure will discourage them from participating in the future.

Admittedly, the above is a contrived example, carefully constructed to support my point (and as an means to work the above quote into my post). That said, don't let that detract from the point itself ... Application of the 'Module Duplication' rules needs to be flexible enough to accomodate the uniqueness of every situation; and recognize up front that every situation is unique.

Proposed Solutions:
In phrasing the "purpose of the 'Module Duplication Check'" at the top of this page, I choose my words carefully ... The goal is to 'Reduce' the proliferation of duplicate modules, not 'Eliminate'. We want to prevent having 17 different ways of doing the same thing ... but I see nothing wrong with having 2 or 3 slightly different approaches to the same use case. In the quest to eliminate the confusion introduced by a proliferation of similar modules, we must be conscious that we do not also eliminate the benefits introduced with 'flexibility of choice'.

The desire to prevent a proliferation of modules needs to be carefully balanced against the need to support/encourage innovation. In my research for this post, I came across a number of community quotes which illustrate this much better than I could alone ...

Let's not stigmatize duplication. It's a source of innovation and a way of exploring approaches.
-Alex_b|

We can all be thankful for CCK's "duplication" of Flexinode. Or imagecache's "duplication" of image module. Innovation has helped Drupal become what it is now.
-Laura S

The "paralysis of choice" scenario is clearly one that is problematic for you. It's not that problematic for me by comparison with the problem of not having options.
-escoles

I would rather six duplicate modules, then discourage a new module developer who may, someday, create a unique Drupal module all of us won't be able to live without.
-shelleyp

Proposed Solutions:
Enforce the duplicate module check ... but be flexible in that enforcement; support the exploration of new or innovative approaches to solving user problems.

In some cases, a proposed module comes along which is a strict, lightweight subset of functionality already provided by another larger, heavier module. As in the 'Barrier to Innovation' topic above, I see this as providing 'flexibility of choice' to end users ... sometimes all that is needed is a 'simple' solution; where end users don't need all of the additional overhead and complexity built into the heavier module. The following comments come from different sides of the debate on dman's 'Open Letter' post:

A lighter, simpler approach may be easier to implement and have lower overhead. ...then there's the [also D5] example of TAC and TAC_Lite: You needed TAC to do some sophisticated things, but TAC_Lite was much easier to use and had fewer side-effects.
-escoles

Making a light version makes sense as long as it stays light. There's a clear difference there that can be explained on the project page. But, for example, making a module with the same features as another because you didn't care for the coding style of the original is not good for the end users who don't care about the code as long as it works and is secure.
-Michelle

... is one solution that's fairly sub-optimal for most things it tries to do, versus a lot of little solutions that are highly optimal for the one use-case they were developed to address. From the perspective of someone who has to maintain the site after I deliver it -- someone I have to train on using it -- a simple, specific solution is generally preferable to a complex, generalized one.
-escoles

Proposed Solutions:
Accept 'lightweight' and 'low overhead' options for existing 'heavy' modules ... but only the 'first' one. (And validate that it's actually the applicant's code, not simply ripped from the heavier module!!!)

This is another aspect of of the problem. In a lot of cases, an author wants to get his project on d.o. mostly to beef up his/her resume. They need something that they can point to and say, "I did that". Sandbox projects take care of that problem to a certain extent, but not to the extent of full project status. So if an author creates a project that duplicates what somebody else has already done, that actually says something about their level of skill. Apparently they do not yet understand that an important part of any project is researching what is already out there, seeing how things have been done by others, and deciding whether starting a new project from scratch is really warranted. There are instances when a new project can be justified, even when there is already something out there that does something similar. In those cases that author really needs to provide some good reasons why the project is necessary.