Success Metrics and Survey

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!
Posted by leisareichelt on April 25, 2011 at 12:48pm
Last updated by tvn on Mon, 2012-02-06 15:39

As previously discussed, it's important that we monitor whether or not our work here is having a positive, negative or any impact at all.

There are two ways we will do this:
1 - site metrics. We'll grab a snapshot of certain site metrics at regular intervals (quarterly) and compare
2 - survey. We'll take a regular survey to capture attitudes to contribution to see if these change over time.

Survey:

I've created a survey here.
Can you please take the survey and give me any feedback on the questions/answers before we distribute this more widely.

Site Metrics

Drupal.org metrics can be seen at http://drupal.org/metrics.

Here are some metrics that would be great to capture for the six months ending 31 March 2011 (in anticipation of quarterly snapshots)

Participation

Total no. Drupal.org members who have logged in at least once (this is the count the Drupal.org front page uses):

Today (May 23, 2011): 592521

SELECT COUNT(uid) FROM users WHERE status = 1 AND login > 0;

March 31, 2011: 564507

SELECT COUNT(uid) FROM users WHERE status = 1 AND login > 0 AND created < UNIX_TIMESTAMP('2011-03-31 00:00:00');

(28,014 new members since March 31.)

Total no. of Drupal.org members who have been active in the last 6 months (active = posted to issue queue, discussion forum, updated documentation, posted to GDO, committed code, etc.

This one's a bit trickier, because that information is kinda scattered all over the place.

This is the number of users who authored either a comment or a node. This would catch discussion forum and issue queue activity. It would not catch documentation revisions, GDO, or code commits.

Total: 16,332

SELECT COUNT(DISTINCT u.uid)
FROM users u
LEFT JOIN node n ON u.uid = n.uid
LEFT JOIN comments c ON c.uid = u.uid
WHERE u.login > 0
  AND (n.uid = u.uid OR c.uid = u.uid)
  AND u.status = 1
  AND (n.status = 1 OR c.status = 0)
  AND (n.created BETWEEN UNIX_TIMESTAMP(
    DATE_SUB(NOW(), INTERVAL 6 MONTH)
  )
  AND UNIX_TIMESTAMP(
    NOW()
  ))
  AND (c.timestamp  BETWEEN UNIX_TIMESTAMP(
    DATE_SUB(NOW(), INTERVAL 6 MONTH)
  )
  AND UNIX_TIMESTAMP(
    NOW()
  ))
;

(This query also takes like 2 minutes to complete, and I welcome a more performant/accurate one. :P)

Collaboration

Of issues posted in the past six months:

Total: 61019

SELECT COUNT(n.nid) FROM node n WHERE n.type = 'project_issue' AND n.created > UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 6 MONTH));
  • Total open issues

Total: 18558

SELECT COUNT(n.nid)
FROM node n
INNER JOIN project_issues pi ON pi.nid = n.nid
WHERE type = 'project_issue'
AND n.created > UNIX_TIMESTAMP(DATE_SUB(NOW(), INTERVAL 6 MONTH))
/* 1 = active, 4 = postponed, 8 = needs review, 13 = needs work, 14 = rtbc, 15 = needs porting */
AND pi.sid IN (1, 4, 8, 13, 14, 15);
  • Total issues resolved in past six months
  • Average no. people contributing per issue
  • Average comments per issue

This (thanks, mikey_p!) will get you a big "spreadsheet" of issue numbers + number of comments. Need to parse it down into averages, and filter by date:

SELECT pi.nid as nid, count(c.cid) as count FROM project_issues pi LEFT JOIN comments c ON c.nid = pi.nid GROUP BY pi.nid ORDER BY count DESC;
  • Average duration (time from date opened to date closed)
Categories:

Comments

Unless I'm doing it wrong,

Posted by davidhernandez on April 25, 2011 at 1:15pm
davidhernandez's picture

Unless I'm doing it wrong, the last part of the survey seems to not be working right. It only lets me answer 4. The bullets are tied together.

thanks - good pick up

Posted by leisareichelt on April 25, 2011 at 1:43pm
leisareichelt's picture

thanks David, that's exactly the kind of thing I was hoping someone would pick up for me.
fixed that (crazy default) setting now :)

thanks!

leisa reichelt - disambiguity.com
@leisa

The survey looks good

Posted by eliza411 on April 25, 2011 at 3:43pm
eliza411's picture

The survey looks good overall. What I see are all minor things:

  • I had to look carefully at Question 2 to find the training/support option. I might separate them just because I did support for a long, long time before I ever did any training, but they're probably fine as they are as well.
  • There are a couple typos in #5, third answer: "can moreo easily see when thinkgs,"
  • It's too bad the URL on the finished page isn't clickable.

Thanks for all the work you're doing here!

thanks!

Posted by leisareichelt on April 25, 2011 at 4:03pm
leisareichelt's picture

  • I was tossing up whether to separate training and support and decided not to because I wasn't sure how much 'contrib' training there is... I've separated them now tho, so we'll see

  • fixed those typos thanks - and yes, it's a pain but I can't put a clickable link in there... imperfect survey software...

leisa reichelt - disambiguity.com
@leisa

Added some metrics + queries...

Posted by webchick on May 24, 2011 at 6:29am
webchick's picture

I went digging around in the Drupal.org database today, and started filling in some stats and queries you're asking for. Apologies for the rawness; I want to putter away on some kind of sweet dashboard-y thing like Mozilla is doing, but baby steps first. :) I'm going to ask around about how to fill in some of the stuff that is still missing.

Only a minor wording point, I

Posted by naught101 on May 24, 2011 at 6:42am
naught101's picture

Only a minor wording point, I consider myself an active (or at least, intermittently active) member of the developer community, but I wouldn't really call myself established. That seems a bit pretentious for someone like me..

retrospective stats

Posted by catch on May 24, 2011 at 6:51am
catch's picture

I had a look at this for another community site, but I don't think I kept the actual queries anywhere useful, were quite simple though - was a quick afternoon's exercise.

It was interesting to compare a few things year by year - it's easy to do that retrospectively - in this case it was for 2005 to 2010:

  • number of non-blocked users created in each year.
  • number of comments posted in each year.
  • unique users posting comments in each year
  • unique users posting nodes in each year

You can also compare the unique users in each year to the unique users for the same query over the five year period - that tells you roughly how much turnover there is year on year (i.e. if 1,000 people post a comment on average for five years, that is going to be somewhere between 1,000 and 5,000 unique people over the whole period which is a massive difference).

Another thing we did was

Posted by catch on May 24, 2011 at 6:52am
catch's picture

Another thing we did was segment posting by node type - i.e. how many unique users post forum topics vs. issues vs. handbook pages.

There's some previous posts

Posted by greggles on June 10, 2011 at 1:14pm
greggles's picture

There's some previous posts on this topic in the marketing group, specifically this post might have some helpful ideas.

knaddison blog | Morris Animal Foundation

Metrics thoughts

Posted by jhodgdon on October 24, 2011 at 7:37pm
jhodgdon's picture

At the GSoC mentor summit this past weekend, there was a discussion on metrics... which didn't seem to be going anywhere, so I went elsewhere. But one thing may be useful - a blog post that lists several projects that have done some interesting metrics things (and there may be more in the comments):
http://iquaid.org/2011/08/12/working-group-on-community-metrics/

Drupal programmer - http://poplarware.com
Drupal author - http://shop.oreilly.com/product/0636920034612.do
Drupal contributor - https://www.drupal.org/u/jhodgdon

Survey results analysed and published

Posted by lisarex on January 24, 2012 at 5:32am
lisarex's picture

Dharmesh Mistry and I did the analysis and written summaries of Leisa's Prairie survey, along with the Getting Involved survey we published earlier this fall. The results of both:

Executive summary:
http://drupal.org/node/1399056

Detailed analysis and findings from the Prairie 2011 survey
http://drupal.org/node/1399258

Detailed analysis and findings from the Getting Involved 2011 survey
http://drupal.org/node/1407270

==================================
http://about.me/lisarex

Drupal.org metrics is now

Posted by tvn on February 6, 2012 at 3:38pm
tvn's picture

Drupal.org metrics is now live at http://drupal.org/metrics
as a result of this issue: http://drupal.org/node/1182998

Adding Metrics

Posted by mgifford on March 6, 2014 at 3:41am
mgifford's picture

There are a bunch of other options here too:
https://drupal.org/node/2193959

--
OpenConcept | Twitter @mgifford | Drupal Security Guide

Prairie Initiative

Group organizers

Group categories

Prairie tags

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds:

Hot content this week

See all hot content.