Applicant surveys (from April to August 2012)

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
patrickd's picture
Posted by patrickd on October 16, 2012 at 5:05pm

klausi remembered me about the surveys we prepared for the talk at dc munich and yes, it's about time that I publish them as promised.

Also, as the "the process sucks"-discussions are on fire again some of these comments might be useful.

Here are the (anonymized) exports of webform.com:
(tried to import them on google docs it did a horrible job, I recommend you to use open/libre/whatever-office and set "tabs" as delimiter.)

Survey 1:
"Project application process"
30. Apr - 14. May
122 Submissions
http://groups.drupal.org/files/pasurvey1.csv

Survey 2:
"Project application queue - survey"
4. Aug - 20. Aug
55 Submissions
http://groups.drupal.org/files/pasurvey2.csv

Sorry for the delay and my absence in the current discussions.
Best regards

Comments

This is seriously amazing

Posted by greggles on October 17, 2012 at 6:54pm
greggles's picture

This is seriously amazing information!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Starting with survey 1, some items I noticed:

  • 55 total respondents
  • 54 out of 55 are satisfied or very satisfied with the process (!!!)
  • 60% were satisfied or very satisfied with the speed of the program
  • 73% participated in the review bonus program
  • Of the people who didn't take part in the review bonus, 53% were unsatisfied with the speed of the process. Of the people who did take part, 67.5% were either very satisfied or satisfied with the speed of the process.

In terms of which measures they felt were most or least important:

Label Git/repository Duplicating Licensing Documentation Coding standards API usage Security
Not important 1 1 1 3 1 1 1
Least important 7 7 8 9 7 0 0
Important 36 29 34 28 29 20 12
Highly important 11 18 12 15 18 34 42
Mean 2.04 2.16 2.04 2.00 2.16 2.58 2.73
Median 2 2 2 2 2 3 3

For the mean and median I just replaced the text labels with a 0, 1, 2, 3 respectively.

So, it seems like people agree on API usage and Security, but are less enthusiastic about the others. It's interesting that everyone thinks there is some importance to all the checks. People seem to think licensing is not a big deal, but in my opinion that's a pretty huge deal.

I hope to do more analysis later (especially on the other survey). Thanks for making and sharing the survey results.

knaddison blog | Morris Animal Foundation

Ok, more analysis.Of the 55

Posted by greggles on October 18, 2012 at 3:43pm
greggles's picture

Ok, more analysis.

Of the 55 people in survey 1, when asked "How much do you think you've learned while the process?" 65% said "very much" 24% said "a little" and the remaining 11% were evenly split between "nothing but coding standards" and "nothing at all".

Most of Survey 2 seems to be similar to the other data. One interesting pair of questions: "How many weeks have you been in the queue?"? AND "What is the maximum count of weeks the process should take?"

Average type How many weeks have you been in the queue? What is the maximum count of weeks the process should take?
Mean 11.18 86.78
Mean (3 removed) 11.18 3.7
Median 8 2

There are a few outliers that are really high in the "maximum" column (52, 99, 9999) that are probably best excluded. If we exclude them the mean drops to 3.7 weeks, the Median of 2 also seems like it expresses that it should be pretty short.

There were 122 respondents in Survey 2. When asked "Do you think the review bonus program makes sense?"
* No 12%
* Yes, but I have no time for reviewing other applications 30%
* Yes, absolutely 57%

I skimmed the comments in both surveys and they seem to have a lot of the same ideas we're working on:
* Automate more
* Integrate more automated review into all project pages on d.o
* "The reviews are important, but I personally don't have time for them"

Very very interesting stuff. Thanks again for doing this Patrick and Klausi!

knaddison blog | Morris Animal Foundation

thanks for this analysis greg

Posted by patrickd on October 19, 2012 at 9:06am
patrickd's picture

thanks for this analysis greg :)

Everyone else interested / wondering what the majority of people think about the process should really read through the comments, there are some very good points mentioned and maybe this will show you that some people have a whole different opinion on how the process is and how it should be.
As I first read through these I was actually really surprised how many people think very positive about it.