I spent quite some time on the project application issue queue in the last months and enjoyed my work on the automated review scripts (pareview.sh and drupalcs). It was a happy day when we reached zero "needs review" applications in November, but I'm a bit frustrated that we are back at 113 issues needing review now. A handful of people is putting quite some effort into the queue, but this is not sustainable and we are at a point where the queue will get out of control again -- soon.
I want to help people that help people. I want to review applications where the applicant has reviewed other applications. I would like to see a queue that handles itself and git admins that are only required to finally approve RTBC issues. We need more reviewers and the pool of applicants seems to me the most logical source that has an interest of getting things done in the queue.
So I will start an experiment: I will only review or approve project applications where the new contributor has done at least three reviews of other project applications. And by "review" I mean a manual review, not just running the automated scripts.
Planned workflow:
* I create a meta issue in the project application issue queue where I describe this idea and where I offer tips for reviewing projects.
* I offer a review in some project application issues where I link to the meta issue.
* An applicant reviews 3 other projects and posts links to the reviews in her own issue summary under "Review of other project applications" (with the exact link to the comment she made).
* The applicant tags her own issue with "PAReview: review bonus".
* I regularly check the queue for this tag, verify that the reviews have actually been done and provide feedback.
* I remove the tag. If the application needs another review later, I want another 3 reviews of other projects (is that too greedy?).
I will of course still look at RTBC issues. And I do not want to enforce this idea on anyone else (so of course you are free to review anything you want).
So what do you think? Is this a bad idea or is it worth trying?
Comments
Off-topic ... but if you're
Off-topic ... but if you're looking to build on the pareview.sh and drupalcs experiences, there's always http://drupal.org/sandbox/jthorson/1367220 and http://drupal.org/node/1328938 ... ;)
And don't let 113 applications get you down. All things considered, the net result last week was a decrease of nearly 25% in the 'needs review' queue.
lost all excitement about
lost all excitement about ProjectApps Customizations as I realized.. damn d.o is still on d6..
that sucks (I came to drupal by d7 and only had a little work with d6 modules [wasn't fun at all]) :/
The testbots are seperate
The testbots are seperate machines ... we could always build a D7 testbot which communicates with the D6 server side (qa.drupal.org). (/end thread derailment)
Hmm.. this reminds on forums
Hmm.. this reminds on forums that have a 'second level' permission, where users are granted to access the second level after posting eg. 150 comments. This should result in more activity but very often to lots of spam.
In our case I'm afraid of kind of 'quick & raw'-reviews possibly stating wrong issues due to the lack of knowledge leading the reviewed applicant to apply false corrections on their project. I'm saying this because I think I've quite good knowledge about Drupal development and its best practices but nevertheless I'm still sometimes failing on reviews.
But sure, we could give it a try and see where it ends, it hardly could be worse than having 200 needs review issues again.
Yes, it may lead to more
Yes, it may lead to more activity, but this is what I want. And believe me I will be strict and will only account for properly done reviews.
I think that reviewing is also an important skill that new contributors should have. Practicing it will help them when they receive patches for their own projects and it will also be a nice learning experience for everyone involved. I think we can live with some reviewing mistakes, the participants will surely learn something in the process.
Another strategy
One thing I've tried a bit with the same goal in mind is suggesting applicants recruit their own reviewers from relevant Drupal communities. A theme applicant might seek reviewers in the design group on g.d.o., or a Commerce applicant might seek reviewers in the Commerce group. This, at least in theory, brings in more capable reviewers, both because they have more experience in the community and because they have knowledge of the specific topic needing review. I've only seen a couple new reviewers from doing this, and I'm not sure if any of them stuck around, but I think it's worthwhile even if it only spreads awareness in the wider community that reviewers are needed.
That said, I'm in favor of trying new experiments to see what works.
Yes, that is also a good idea
Yes, that is also a good idea to reference (sub-)communities that might have an interest in a particular project. This is also no contradiction to my proposal, we can do both.
I think it's a good plan
Klausi,
I think it's a good plan. Thanks for your review today. I don't want to post in my application that I reviewed another person's project since another reviewer said not to post to my own issue unnecessarily, since other reviewers might think someone else is working on it. Anyhow I think this will work to encourage applicants to review other applicants only if on the issue page it makes it prominent that the applicant's next review will only happen, or will happen faster if they review at least 1 -3 other issues.
I've been monitoring what you guys are up to and saw this thread, and so reviewed another application just now: http://drupal.org/node/1166144#comment-5482326 I am an applicant that wants to get involved either in helping with core, or with this que. My Droogle module is just really my attempt to get involved more, although Droogle will of course be supported -- it's a module Babson College is asking me to maintain anyhow. I digress.... anyhow if you don't make this prominent on applicants issues they will not find your thread on this. I actually would suggest a block or 2 on the issue page itself for applicants with 1.) info on how to speed up their application with links to pareview, links to the coding standards, etc. 2.) in a block perhaps state that preference will go to applicants that review another application. There are also scoring modules in drupal as you know that could be appropriate -- but there are many ways to skin this cat.
I do like your idea. I've led large groups in a "prior life" and delegation, and pyramid type setups do work, when the spirit of cooperation is made clear so folks understand why they are being asked to help.
Anyhow I'll go do another review. And patrickd, I hear your concern about folks doing a bad job reviewing. I am new and have nervousness about my reviews, but life doesn't happen without taking a step forward, even if there is the fear of failure. But I agree reviews that aren't thoughful that just try to play the system are no good. My guess is, as a whole applicants are a fairly thoughtful group, that want to learn and grow in their drupal careers. Like me many are just trying to get more involved, see who they meet, network, learn from other high level folks, contribute to open source, contribute to society, etc.
Anyhow Klausi, I definitely liked your comment. And yes I need a review, but heck no matter what I say while needing a review I will sound like a beggar, so I will ignore my lowly status!
on applicants issue a block should also have a link to the que!
I'd like to add I just went looking for the que (of module applications) itself and it took me a few clicks, and in the past I haven't gotten there fast. I just googled it btw, still couldn't find it, so then on my issue clicked project applications and then finally found the link: http://drupal.org/project/issues/projectapplications?status=8
Perhaps my eyes are not great (which they aren't) but adding to my idea above if you put a block or 2 on applicants issue page, a link to the que itself would go a long way toward giving folks an easy start in helping out: http://drupal.org/project/issues/projectapplications?status=8 I just double checked, there is no easy click to the que. This is bad design if I were designing drupal.org specifically for promoting the que. Some might argue that folks should want to review others, or how serious are they about it. Still not good design -- applicants are busy, have full time drupal jobs, want to contribute, but workflow should help not hurt our busy community.
Anyhow I'm sure you guys discuss good design for your web pages all the time to draw eyes and activity in a desired direction.
Per default a user has this
Per default a user has this "Contributor Links" widget on his dashboard.
I think this is enough linking
Starting...
Experiment is now officially launched: http://drupal.org/node/1410826
Good going klausi! Seems to
Good going klausi! Seems to be working quite good, eh? :) At least I got a lot more motivated to do some reviews to get my own module reviewed, and I'm going to continue do some reviews when I find the time.
The hardest part was to get started doing reviews, but after I'd done a few I noticed it isn't as hard as I thought and it feels good to be able to help people.
How do you feel about the issues that have the PAReview: review bonus -tag? Do you want help reviewing those and spreading the word about how to get the bonus tag?
Continue with the good work!
Yes, I'm quite happy how it
Yes, I'm quite happy how it works out. Feel free to review applications with the tag, I even encourage it in the review bonus outline. Let's keep those applications busy that help others by reviewing code! And of course you may spread the word about it :-)
existing modules
Perhaps a dumb question, but what about reviewing existing / approved modules ? IIRC there is a project for reviewing core modules with PAReview, but what with all the other modules ? As a happy Drupal user, I could spare (and justify) some time improving the (full project) modules my department is using.
You can of course also review
You can of course also review existing projects, just create a new issue in their issue queue where you report the things that you find.
Both activities are welcome.
"Code review" is a process which I feel belongs in the "new contributor applications" area, but I think that existing projects wouldn't really benefit from a code review process. They do benefit from a patch contribution process. Both activities (project-application reviews and project patching) are quite welcome.
I'd even suggest that my personal thought on the matter of improving existing projects is that identifying patchable code via a code review is still less useful (to the maintainers and to the community) than reviewing the issue queue and identifying patchable code that way.
My personal opinion is that,
My personal opinion is that, if one has time to spare, helping onboard new contributors (though the work done in the project application process) provides greater long-term value to the overall community than supplying coding style patches to existing projects.
That said, the great thing about the Drupal do-ocracy is that my opinion doesn't really matter ... if you find something you want to do, and see value in doing it, then by all means ... just do it!!! ;)