OK. So I know this is a big hairy, scary one, but I think it needs tackling.
We need a way to better recognise a wider scope of contributions to Drupal for 'social' reasons (so people who don't actually know each other can more easily determine who is who and how they contribute to the project) so that people who don't actually 'commit' anything but contribute a lot have their contributions more recognisable, and then so that we can potentially match people to projects/issues when we need people to help!
We could go crazy with all kinds of complicated reputation systems but I'm proposing that we keep it pretty simple and human driven (I think this is a problem that is better solved with people than technology).
Here's what I'm thinking (and I'll try to post some sketches of this shortly too).
-
we develop a taxonomy that lists our areas of expertise which could be visual design, theming, user experience, tech writing/documentation, coding, testing, reviewing, pls feel free to add/edit/etc.
-
we map that taxonomy to 'phases' of a project (ref: the wireframe that I did for the issue template and the different tabs/phases that a project goes through http://bit.ly/hzLsZh ).
-
terms from the taxonomy can then be associated with your profile on Drupal in one of 2 or three ways. Firstly - you can say that you have expertise in one or more areas of practice. So, I might say that I have expertise in UX for example. Secondly, someone else can look at your project an say that you have expertise in a particular area of practice. So, I might look at your profile and say that you're good at documentation. If I do that, then it will say on your profile that I think you're good at documentation, so this is more than a 'quantifying' tool - it's not necessarily about how many people think you're ace at documentation, it's more about who think's your good at documentation. This will have relevance either if I already know someone who is vouching for you and I trust their judgement OR if I know people who recognise good documentation when they see it and I see they are vouching for you. (These are obviously not mutually exclusive). This means that we will hopefully be more thoughtful about who we vouch for because we're kind of putting our own reputation on the line by doing this. (You don't want to be seen to be vouching for someone who is incompetent or a douchebag, right?) I'm drawing a little on Quora again thinking about how they show who has +1ed answers on Quora not just how many people have +1ed.
-
terms/areas of practice can then be given more or less prominence by people in the community +1-ing you for that area of expertise. So, for example, I'd go +1 the people who I know who already contribute a lot to UX in Drupal on their profiles, assuming they'd already 'tagged' themselves (eh, I know, I'm mixing tags and taxonomy... this is why it's not good practice to describe implementation as you're describing your design idea!). (See above re: trying to make it not about numbers but more about who is vouching for you).
It's as simple as that. I did think about mapping actual activity to 'terms' but then thought how do you distinguish between good activity and well, less productive activity. It gets complicated fast.
I think what I'm describing above is a great way to take a first step into this territory and then we can gradually tune/experiment with the system over time to see how we can add to it without turning it into a game.
The side effect of this is that we can then use this mapping to put a call out to specific people when we need them to help on a project with out having to know who they are and hope they're in IRC. It also gives newbies and easier way to start getting involved in contribution.
We'll also need to look throughout the site to see where we need to remind/prompt people to acknowledge expertise and find a way to do this so that it's as easy as possible.
Thoughts?
(Edited to add notes re: showing who vouches for you and making this more prominent than how many ppl have vouched for you. So it's not anonymous +1s and less gameable/quantifying)
Comments
I could see this system
I could see this system getting diluted pretty quickly and/or turn into a popularity contest which rewards the rich. Still, might be better than nothing. One thing we have stats on but don't really acknowledge is the posts someone makes on a give project in the issue queue. That is a good indicator of involvement. +1 point for starting a new support request, +3 points for filing a new bug, +5 if it gets fixed (as opposed to duplicate or outdated). +5 for reviewing code, +5 for answering someone's support request, etc... Again, can be gamed.. Still commits are like this too. Some people make lots of small commits to inconsequential projects. Others make a few big ones to well know ones. Tough problem to solve... every time I think about it I get disgusted with the need to quantify people. Wishing we had more evolved ways of communicating value and building community.
Best,
Jacob
using people-ness to reduce gaming
so, obviously I considered the kind of activity based reputation that you're suggesting here... in order for it to be useful, I think a human has to say 'that was a useful contribution', otherwise reems of rubbish is more valued than one incredibly insightful contribution, which is just not true.
this is why I think the people thing is more useful. what I should have mentioned in my original post (will go back and edit) is that we should show who has said that you have expertise in a given area of practice. This makes it less easy to be gamed and it makes you more thoughtful about who you're 'recommending' as experts in various areas (because people will be able to see that you said X was a good reviewer, for example).
It could introduce some interesting new social behaviours... in fact, I'm sure it will....
but not only is it easier to design/implement, I also think it will enable us to reward the kind of contribution/behaviour we want to reward, not just getting busy doing anything.
I like to think about it less as a method of quantifying people and more as a way of visualising what already exists in the community - connections between people you know who recommend people you don't know, with the side effect of helping us to find people who could help us who we don't already know.
leisa reichelt - disambiguity.com
@leisa
Like
I like what I'm reading. The idea is getting better. Will think about this and post back if I have something to add.
Oooo, now this I like!
Good call, Leisa! This is good stuff! I +1'ed it.
Joel Farris | my 'certified to rock' score
Transparatech
http://transparatech.com
619.717.2805
Slashdot-like system?
How about translating these ideas to a Slashdot-like 'karma' system using the 'tags' Leisa described? This way the recommendations are linked to the actual activity in the issue queue (like Jacob suggested), but you also have the 'people thing'. You could have things like:
+1 helpful (answering support request)
-1 unintelligible question (support request)
-1 duplicate (that'll teach you not to search first ;))
+5 patch
+2 testing
-2 help vampire
+3 documentation
+4 code review
etc. And the accumulated stats could be shown on the user profile.
I think there will be a lot more adoption if the system is right there with the activity, and you don't have to go 'out of your way' to use it.
This also makes it more attractive for the large group of users who are not module developers, themers, active documentation contributors, etc., to participate. For example, I'm not active in any of those three areas so I wouldn't be likely to be ever vouched for as a specialist in some area.
But I can (and do) answer support questions in the issue queue, test patches, try to troubleshoot a problem, etc. With the community growing ever larger, I think the impact of encouraging many tiny contributions could also be very important - in addition to recognizing more substantial contributions like Drupal core and contrib code, documentation, UX work, etc.
Definitely need in context prompts
So, I totally agree that we need to design in prompts for people to show recognition with out having to remember to go elsewhere. I'll add that to the original post. I also think that the taxonomy of expertise has to include all the kinds of behaviour we want to reward/show we value. So, as you say, providing support is totally an area of expertise! ( perhaps we need a better word than 'expertise' to describe this?)
I'd rather start as simply as we can and add in more complexity as it's required - so down the track we might need a more granular 'points for activity' system, but those are so easy to screw up... Makes me nervous, and I suspect that the simple version might address a whole stack of problems and be much more palatable to the community at large.
Thoughts?
leisa reichelt - disambiguity.com
@leisa
Well, we already know we need
Well, we already know we need some kind of point-based system for the support.do project, so I'm sure we'll be a test case for this.
This is a good idea.
This is a good idea. Once we've nailed down what we want to reward/give points for, we should get a prototype system in place to test our theory. We aren't going to know how it'll work until we test out a point based system. What kind of metrics would we use to see if it's working as intended?
http://www.webbfx.net/
Slashcomments
http://drupal.org/project/slashcomments
what you don't know will inspire you
very nice! I assume we're
very nice! I assume we're talking about contributions/experience giving back to the community versus "what types of work" someone does?
if we're trying to recognize giving back to the community, then I have a couple thoughts, mostly centered around I like the idea of encouraging and supporting involvement by recognizing it. but I think the language needs to be a bit more abstract or more encompassing.
for example: to your first bullet point about expertise, (#1) I'd suggest adding more community related expertise/involvement. (event planner, drupal evangelist, association board member, public presenter, emcee)
continuing,
You didn't link to your wireframe for the issue template, so I have no reference. But I'd suggest at least one 'phase' would need to be added to include 'community activities' (meetup event organization, presentations, users helping users, etc).
lastly, in bullet 3, you say
my concern is the use of "project" in this idea
again, I'd like to see a wider way of recognizing contributions. maybe thats what you meant... and obviously contributing code is important, but by far one of the current shortcomings is not adequately recognizing people that contribute in ways other than code. your example even used documentation as an example, but that seemed unclear to me, and makes me wonder what else might be in the list?
--
mike stewart { twitter: @MediaDoneRight | IRC nick: mike stewart }
Issue page wireframe is
Issue page wireframe is located here: http://bit.ly/hzLsZh
Newbie opinion
As a relative newbie to this group and a light dabbler in the Drupal community so far, I don't have the benefit of experience of how people who've seen a lot and done a lot here see things, so there might be a bit of a disconnect here in what I'm writing and what makes most sense.
In terms of someone's reputation, I don't think a point-based system necessarily has much to offer in terms of value, unless it's broken down into the different areas of expertise (somehow) in order to quantify it in some meaningful way. Someone committing a small but significantly valuable set of Field API changes won't necessarily have as much in terms of points as someone who's replied to a whole host of issues, whether their input was valuable or just a "+1" post, which might distort the whole picture. It's also open to gaming as already mentioned, someone could boost their score by carrying out a long stream of meaningless tasks, meaning there would have to be a consideration of consequences for those who do so.
Coming from what I have experience of, I think a system like LinkedIn's recommendations has some merit, whether this is an extension of or in addition to the vouching system that Leisa has mentioned. If someone had a link alongside their name listing the number of recommendations/vouches they've had and the ability to drill down and find out what these were for, I think this would add much more value.
Also, in terms of recognition of regular contributions (rather than points), I think that the current situation of seeing these people's names everywhere across the community is the best. I can list around 20 names of people that I know have fingers in most pies, and I'm not sure there's a way to show that properly on a profile (or to be more precise, those many thousands in between) without info overload.
Points + something else?
In other large projects, how have point systems worked alongside the power curve which is ostensibly important to FOSS? eBay combines several metrics to create a system that rewards people who participate at different levels. Stars indicate level of participation (# of transactions), but percentages can still be useful to establish strong credibility with lower numbers of sales. I don't participate in any communities/forums that use rep, so I'm a bit ignorant on this, but do work in an organization that values accommodating these varying levels of contribution.
Three issues
I wholly agree with the taxonomy of karma/props in the original post.
I like ArjanLikesDrupal's suggestion of a slashdot-like system (which i replied to)
If we could combine those two somehow, I predict shangrila. Well, not really but you get my drift.
I would also add the notion duration of reputation based on http://drupal.org/project/radioactivity. I think this is important because we want to make sure that newer contributors feel the love while missing persons fade into oblivion. Sure that's a little exaggerated, but you get the drift.
And if we have a list of who vouches for you, couldn't that inspire gaming the system? As long as that list isn't ranked, otherwise we'd be building a google of reputation (which on second thought doesn't seem to be that bad of an idea).
what you don't know will inspire you
A much more concrete plan of
A much more concrete plan of attack has emerged in http://groups.drupal.org/node/209448, go check it out!