Review Process - Basic Steps

Events happening in the community are now at Drupal community events on www.drupal.org.
ccardea's picture

One of the things zzolo talked about in his post was formalizing the code review process. Under that heading, he included describing the basic steps of the process and the outcomes of these steps. I put together an outline of basic steps, but I would like to get some feedback before putting it on the wiki.

Basic Steps

I've divided basic steps into two categories, based on the level of expertise required to get the job done right.

Initial Review

These are steps that can be performed relatively easily, do not require a high level of technical expertise, and should be completed before technical review.

License review

Check files for non - GPL2 content, primarily 3rd party libraries and possibly image or other files.

Module duplication review
  • Search for modules that provide similar functionality.
  • Ask the project owner what makes his project different from others that provide similar functionality.
  • Consider whether the project should really be a feature of an existing project.

Technical Review

These steps require a higher level of technical expertise.

Drupal API

Demonstrating a good understanding and use of the Drupal API's is high impact. The project should not short-circuit the Drupal API in ways that could cause Drupal not to function properly or not inter-operate properly with other modules.

Security Review
  • TODO - looking for some specific suggestions from more experienced developers
Coding Standards & Best Practices Review

Coder module is available to help with this. Coding style issues are not critical and can be cleaned up after the project has been approved.

  • TODO - other
Documentation review

Check for both user documentation and developer documentation. Documentation issues should not block approval.

  • check for README.txt and read it.
  • Has hook_help been implemented?
  • review Doxygen style code comments. This is not as important for contributed modules as it is for core, unless the module provides an API that is intended to be used by other modules

Comments

There's already documentation

greggles's picture

There's already documentation at http://drupal.org/node/894256

Is this mean to replace that?

Audience of the handbook page

bonobo's picture

I'm getting a blog post together about code reviews (and the Portland DUG will be running code reviews at the next meeting), and the thing I noticed about the handbook page at http://drupal.org/node/894256 is that it's split between two audiences: those reviewing apps, and those looking to get their apps approved.

The title and the first part of the page seem pretty well suited for people reviewing applications.

Starting at the Workflow section, the page shifts focus and becomes more relevant for people looking to get their applications approved.

I'm thinking that this page could be split into two pages; one for reviewers, and a second for applicants.

But in any case, having some explicit instructions to structure code reviews would be helpful for everyone. There's an additional wiki page in progress that covers some of these details. Centralizing these docs in one place would be ideal.

Audience of Handbook Page

ccardea's picture

This is an important point and I agree that there really are two audiences, and it probably would be beneficial to have pages specifically addressed to each group. I was talking to zzolo on IRC and we both agree that we need to define a basic standard of quality for contributed modules. It is then the job of the project author to adhere to the standard, and the job of the reviewer to provide a level of assurance that the standard has been met. I'm thinking that there could be three pages, one to define the standard, one for applicants, and one for reviewers.

Thank you for pointing me to your wiki page. I will post some comments there.

Handbook Page

ccardea's picture

I just took another look at the handbook pages, and there already are two different sections, one for applicants and one for reviewers. It might be possible to move the Workflow section of the reviewer page to somewhere on the applicant page. This is really a documentation issue.

Formalizing the Review Process

ccardea's picture

In zzolo's post, which you can read here, he talked about formalizing the review process. This is meant as a first draft documenting the basic steps of the process, with some specific things that should be done. I probably need to give a little thought to the desired outcomes.

Security Review Objective

ccardea's picture

I'm still thinking about formalizing the review process and gearing up to attempt a security review. I will say at the outset that I feel a little under qualified to do this, since I'm still wet behind the ears as a Drupal developer or any kind of developer for that matter, but if we are going to make a dent in the backlog of project applications, somebody has to do these reviews. People reading this should look at these posts as a learning experience for myself which hopefully will translate into something of value for others as well. Before I put too much effort into this I want to make sure that I'm in line with what the more senior people think, so this post concerns the objective of the security review. Do we simply want to take a quick look to see if the author knows about and is following secure coding practices, or do we want to carefully scrutinize the code and look for anything that the author might have missed?

Security Review Objective

ccardea's picture

It looks like I have to answer this myself, but i feel somewhat qualified, based on my experience as an auditor. There are a few basic principals that apply:

  • Identify where things can go wrong
  • Are there controls in place?
  • Are the controls being used?

In auditing, there are two types of controls,

  1. Prevent controls are controls that prevent errors from occurring.
  2. Detect controls are controls that detect errors that have occurred so that they can be corrected.

In auditing, if you can establish that good controls are in place and are being followed, you can cut down on the amount of time that you spend checking for things that may have fallen through the cracks. Unfortunately, in website security, once a breach occurs, it's too late, so we have to rely exclusively on prevent controls. Fortunately Drupal has done a good job of identifying where the problems can occur and putting controls in place. The code review itself is a form of prevent control, to make sure that procedures are being followed.

It's fairly easy to set up mechanical checks for common items that follow a predictable pattern. But you can not rely completely on mechanical means. You need to be aware of where problems can occur. Where is the project getting data from user input and where is it being displayed? What other types of things can go wrong? It's that type of awareness that will allow you to catch things that the mechanical checks won't catch.

I didn't answer your initial

greggles's picture

I didn't answer your initial question because it takes a long time to answer...in fact, I'd say it takes over 240 pages to answer the question ;)

Basically the task of reviewing a module is to digest all these resources and more:

The perspective you provide above about looking for where data arrives and seeing where it is used is completely accurate.

Thanks, Greg

ccardea's picture

I didn't know that you are a security guy. I'll have to read your book.

From the perspective of doing code reviews, Writing Secure Code is the main focus, and trying to come up with some basic procedures for checking compliance with controls. I noticed that for Drupal 7, Coder is doing more security reviews. I'm a little leery about relying on Coder too much without knowing what's going on under the hood. I've been doing things like using grep to find all sql queries, all form title and description attributes, etc. I'm a little concerned about losing the human factor by relying too much on software. I noticed that you are using Coder in your own security reviews. I'd be interested in your opinion of how effective it is, what it does and does not do.

From the perspective of a git administrator, I would be worried about people saying that they did a security review when they really didn't do much. In a formal audit situation, you would be required to document every procedure that you did and the results. That's a little too much to ask of a volunteer, and I would be interested in hearing from administrators what kinds of things they like to see that gives them some confidence that the security review was done in a reasonably competent way.

We use coder and other

greggles's picture

We use coder and other automated tools in our security reviews and find them to be helpful. But we still also suggesting doing manual reviews.

For this project application review process, I think we shouldn't try to find 100% of the security bugs. That's impossible. We should just look for major bugs and best practices. We should review the code to figure out "does this person generally understand Drupal's API."

If you look across contrib today you'll see that's the typical level of quality.

I get that

ccardea's picture

What I'm trying to come up with is some reasonably efficient techniques or procedures that can be used to conduct a review. Ideally a procedure would do two things:

  1. Provide an objective means for the reviewer to make a decision, and
  2. Provide some evidence to the git administrator that the reviewer did a good review.

I'm still not sure exactly what Coder does and does not do, but it looks like that's the best tool available for Drupal 7.

Provide an objective means

greggles's picture

Provide an objective means for the reviewer to make a decision, and
Provide some evidence to the git administrator that the reviewer did a good review.

Realistically I don't think you're ever going to get there...

I used to work on audits as well (from the technology side, not as a CPA) so I know the mindframe you're coming from but I just don't see it applying to our processes in a reasonable way.

Provide an objective means

ccardea's picture

I think I might be able to come up with something. The key words are realistic and reasonable. I'll get back to you when I have a few more reviews under my belt.

Coder is for review the code

rteijeiro's picture

Hi.

Coder module is the best tool to a fast and easy code review. It helps developers to write clean, well organized and well commented code.

I think that code standards are very important if we want others to understand our module coding.

I have just started to review some modules and if the module has a clear code it's easier for me to understand what the module does.

Kind regards.

Calm Down & Clear the Cache

Copyright Notices

ccardea's picture

I was re-reading the GNU GPLv2 license text, and noticed this under the heading of how to apply the terms of the license to your program.

If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.

To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found.

one line to give the program's name and an idea of what it does.
Copyright (C) yyyy name of author

Shouldn't we be advising project owners to put copyright notices in their source files?

No. This is covered by the

tim.plunkett's picture

No. This is covered by the LICENSE.txt file automatically packaged with every module and theme.

In fact, when I see modules that contain author information, I file an issue about it, asking them to remove it. With multiple contributors, and since module maintainership can change, including this is discouraged.

Not so sure about that

ccardea's picture

Here's how the license works

We protect your rights with two steps: (1) copyright the software, and (2)
offer you this license which gives you legal permission to copy, distribute
and/or modify the software.

The only copyright notice in the LICENSE.txt file is the copyright for the license text itself, dated 1989, 1991. I looked into copyright protection in the past, and found out that the only thing you need to do to get copyright protection is to post the notice, but if you don't post the notice, there's no copyright. The copyright is an essential part of the license, so in order for the license to be valid, you have to post the copyright notice.

The Instructions are from the website

ccardea's picture

The instructions are from the website, and you won't find them in LICENSE.txt. I'm sure their advice is valid.

There is a copyright notice

ccardea's picture

The copyright notice is in a file called COPYRIGHT.txt in the Drupal root directory. It reads in part:

''All Drupal code is Copyright 2001 - 2010 by the original authors.''

I'm guessing that "All Drupal Code" covers contributed modules. It's a small detail, but people who are trying to get away with something often do it by exploiting small details. I think that there is really no harm in posting a copyright notice in the source file. It identifies you as the original author.

I do see your point

ccardea's picture

It could get messy if original authors posted their own notices. Technically anybody who changed the code would also have to post a notice. So you're right tim.plunkett. Sorry for the email spam.

Code review for security advisory coverage applications

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: