I'd love to have some kind of metrics that we can monitor to make sure that we're actually improving things through any changes that we manage to push through.
Can you help me work out what numbers we'd want to keep an eye on and then work out how we find out what they are currently so we can start measuring?
Suggested metrics
The kind of things I'm thinking of are:
- We want to see more people contributing. I figure a good measure of this might be the ratio of active contributors (say people who have added a comment to an issue or committed code to core/contrib in the past six months?) to overall Drupal.org members. More = better :) I guess active might also mean 'following' or 'voting' if we add those things into the mix.
- We want to see more people contributing more actively. A good measure might be the average contributions (where contribution = comment in issue queue or code committed) per active contributor per six months?
- We want to see more effective contribution - so perhaps we want to measure the average 'duration' of an issue opened in the past x months (duration being the time between the issue being opened and being closed - I figure this is better than measuring the number of comments etc. because, as long as the issue is resolved swiftly, having more people in the mix and more conversation may be a good thing, right?
(I'm working on a six month model here thinking that people get busy and that doesn't necessarily mean they're not 'active' they just might be working on the thing they're going to contribute shortly? Feel free to amend timescales if you think there's a better measure)
Other metrics?
Are there other things that we could be measuring to see how we're doing and whether we're moving in the right direction?
Qualitative survey
I'm kind of tempted to do a qualitative survey too, a bit of a 'where are you at with contributing to Drupal' to take a broader 'temperature check' among the community and we could re-take that in six months or so to see if we've made anything better.
Thoughts?
Comments
I'd like to see better
I'd like to see better statistics about Drupal core development as well, instead of just number of committed patch contributors (and committed patches per contributor) which used to be just a fun statistic but now gets your mugshot shown on large screens at DrupalCon, which IMO is a bad sign (and I can say this as someone who was in the top ten list for D6 and the top two for D7).
I'm sure there are existing threads about this stuff in the project metrics/redesign groups which might be relevant - although they'll be quite old now.
I'd really like to see the following for the core metrics, I think they'd be as useful for Prairie.
Would be good to be able to do this year-by-year and month-by-month.
If I read the comments under
If I read the comments under the 'painpoint'-post correctly, there's consensus around the need to convert passive and 'semi-passive'* users into active user who dare submit patches, design changes, issue status, fix outstanding hurdles.
In that regard the share of users who go the step being posting 'this works/doesn't work for me' is an interesting metric. That could be (as catch posted) status changes, amount of resolved issues, patches posted etc.
// Jesper Wøldiche
My thoughts: Note that I'm
My thoughts:
Note that I'm coming from the perspective of watching a lot of metrics for a 15 person team for expensive software, so not all of what I'm used/find useful to is potentially useful in the Drupal world. I'll try to weed those out. :D
I like catch's ideas; death to subscribe would probably go a long way towards dealing with #3 there. Especially with the Views Bug Squad - we've added a bunch of people as sub-maintainers, but I want to be able to keep this list current to people who are actually helping. I'd also love to be able to call out the people helping us with concrete "This is how much this contributor has helped us!"
Another one that I find useful is "oldest issue"; there's a lot of cruft in some of the larger queues; there's no current way to find those crusty old ones and get rid of them in an appropriate fashion (unless you have access to the db, which very few of us do). I know the oldest views issue right now is from April 2008. That doesn't mean I can easily find it. We have "last updated", which is useful, but it's not enough.
I personally find the duration metrics really useful; this goes in hand with the one above - older issues are often less likely to be relevant or reproducible in current releases of software. This is definitely truer for some projects more than others (see: views - something found in 6.x-2.9? good chance we won't be able to repro it in -12)
Other things I've found useful:
Number of bug fixes in a release. We sorta have that now with the changelogs, but they're not very user-friendly.
Number of issues a user has had marked dup. Just so I can know who to boot out of the queue. ok, kidding. Sorta :)
More
We could also do things like find out the ratio of people who posted something in an issue queue (ever, in the past year etc), compared to the project usage statistics.
where do we get this stuff?
so, does anyone know who I need to talk to in order to get these metrics?
I'd like to make a short list and get some numbers, like, now! :)
leisa reichelt - disambiguity.com
@leisa
I think list your metrics
I think list your metrics here and see if you can get killes or dww to run some queries.
Might be worth asking Greg
Might be worth asking Greg Knaddison / greggles as well.
Apparently Neil's the Man for the Job
Well, he's already posted a bunch here:
https://drupal.org/node/2193959
I've brought over some of these ideas to that issue queue.
--
OpenConcept | Twitter @mgifford | Drupal Security Guide