Posted by chx on April 25, 2008 at 8:35pm
Today two concepts were raised on IRC: Slashdot style reviews -- but also, Amazon.com also have a "was this review helpful to you" feature. Also we could have "karma" (userpoints) that you can earn by handbook pages, comments here and there, patches, commits... you can spend it on badges and reviews. Very quickly it'd become quite visible who has Level 32 in Module Maintainership :D
Comments
Userpoints
I like the userpoints idea the best. It adds gaming to Drupal.org -- I know I'd spend more time there! I guess that means we'll have Drupal "wizards" that can do userpoint grants, too :P
I like the userpoints idea,
I like the userpoints idea, it'd immediately cut down the issue of one person adding dozens of useless ratings to modules (well, unless someone who contributes a lot decided to do that). I don't see an easy way to grant userpoints though - helping out in the support forums, answering (and providing well written) bug reports in issue queues, helping in #drupal-support - how does this get userpointified?
In addition to
In addition to userpoints/review of modules why not have a Gold, Silver, Bronze status that is earned. These criteria would be predetermined by the drupal admins and would reflect the maturity of the module.
For example, the Gold modules would
1. Be a module that has existed for more than 1 year.
2. Not be a duplicate of any other module
3. Have active developers
4. Relatively active in the issue queue
5. Other modules depend on this module.
etc.
To ease the load on the drupal website admins, most of the stuff should be easy to track and maybe automate some of it.
Anyway. A predetermined set of criteria to give the user another dynamic to evaluate the module.
3 and 4 are discussed here:
3 and 4 are discussed here: http://groups.drupal.org/node/10629 (and I like that idea).
I also like 5 and 2. However 1., I think older modules will get higher ratings anyway.
I guess I envisioned this as
I guess I envisioned this as being complementary to user ratings, not in replacement of user ratings. It's a more objective approach to modules. It's kind of like having a comparison chart as well as user reviews when buying a product.
Could be fun to show off your stats, but...
Adding experience points could upset the relatively level playing field that Drupal.org offers. At the moment, all users are basically equal, not even post count is displayed. If we start ranking users, it must be done in a way that doesn't discourage or belittle those who are new to Drupal. If it leads to something like "I'm better than you because I'm level 32. You're only level 1, so your opinion doesn't really matter", it's probably the wrong way to go.
I'm also wary of Drupal turning into a popularity contest where users compete for cliques of friends who give each other +1s at any available opportunity (or vote down those who they don't like, just to spite them). I think Digg.com is a good example of how this promotes a "lowest common denominator rises to the top" problem.
Designing a fair and useful point system for this would be difficult. Like Catch said, a lot of the effort put in by maintainers and volunteers is not readily visible or able to be voted on. Some things could not be tracked at all (helping on IRC, for example).
As far as "Gold modules", I think Acquia is probably going to set the standards here. They appear to be selecting a group of high quality contrib modules for their carbon project, and those will most likely get some kind of elevated status.
Slashcomments module
I just released a module to implement Slashdot-style comment moderation and viewing.
You may check it out here: http://groups.drupal.org/node/12439
Delf.
Reviving this
I came to this group to start a new discussion on this topic but see there already is an old one.
I think this is an important factor in the redesign. John says "Adding experience points could upset the relatively level playing field that Drupal.org offers" but I would argue that the "playing field" being level is only an illusion. Anyone involved in the community can clearly see that there is a karma system at work and a hierarchy based on the doacracy. The problem is that you need to be involved in the community to see it and it is largely invisible to new users.
I think it's time we made the karma system visible. When you are new to a community, it's helpful to be able to know something about who is answering your questions. Sure, newbies can give good advice and oldies can give bad advice. But overall, someone with more experience is going to give you a better answer. Another part of this is when people are warned about posts. I've warned a few that their posts were borderline spam or for some other reason on the verge of being unacceptable. But who am I? There is no clear indication that I am a site maintainer. Even looking at my profile doesn't reveal this. So why should they care what I say? Now I'm not saying that being a site maintainer makes you god of the forums or anything but I do believe it puts a bit of weight behind my warnings.
On the flip side of this is the incentive that karma can provide. If you know your efforts are being recognized, that's a lot nicer than toiling on in obscurity. Of course this can lead some people towards trying to game the system, which is why I think a karma system would be more effective than a strictly points based one. I don't want people racking up posts for the points but rather I want to see people rewarded for their efforts.
Done right, I think adding visible karma will be a benefit to drupal.org so I hope it will be considered.
Michelle
See my Drupal articles and tutorials or come check out the Coulee Region
But how?
I wonder what criteria you would base karma on. Sheer quantity shouldn't be enough: someone spamming the forums with one-liners shouldn't get more karma than someone who only periodically helps out users but with high quality responses. Also, would you get karma for writing code-standard adhering modules, for following up quickly on bug reports, for keeping your sites up-to-date with the latest stable version of modules?
A simple rating system
I'd like to see a very simple reader-driven rating system so that any registered user can mark posts as "Helpful", "Unhelpful" or "Neutral" (default).
Then I'd like a filter so that I can limit my view of a page to only the posts marked Helpful or to only the people with a track record of posting helpful comments.
That is a good idea
I like the idea of it being based on how helpful something is, but I think there should be some recognition for people that actively participate or have a certain role on the site.
.
As Lee said, Karma would come from other members. It's still possible to game it with shill accounts so we'd need to watch out for that. But it seems to work well enough for other communities. Since Drupal is more than just a forum, we'll need more mechanisms than simply marking posts helpful or not. There needs to be something on the profile page itself where a user can give you karma, maybe with a note about why. Sort of like the eBay feedback system.
While modules like userpoints are cool, I just don't think they're practical for this community. We'd have to figure out how to assign points for all sorts of different ways of contributing. What I'm envisioning is simply making the karma that's already there more tangable. If someone helps you whether it be writing a module that you use, answering your forum post, giving you a tip on IRC, whatever, you up their karma. Simple as that.
FWIW, I don't think there should be negative karma. Let's keep this positive.
Michelle
See my Drupal articles and tutorials or come check out the Coulee Region
Perhaps no negative karma,
Perhaps no negative karma, but a "spam" or "malicious" rating might be useful to report any such posts.
--Andrew
I wonder what criteria you
The number of episodes of "Earl" that you've watched. ;o
Sorry, I couldn't resist.
In case you don't know:
In case you don't know: There is a related discussion on the flickr group: http://www.flickr.com/photos/_leisa/2837345676/in/pool-drupalredesign
.
No, I didn't know. This redesign is spread out over too many sites. I can't comment on the flickr page, so I'd like to keep discussing it here.
I like the "props" idea but I'm not sure on limiting how many a person can give out. I can see making a control so you can't give more than one to a single person in N $time but you should be able to give them to all the people that help you.
Over all, I'm not too worried about gaming here. I think the Drupal community is fairly mature and serious and less likely to do such things. Sure, there will always be some that do but they should be pretty easy to spot.
Michelle
See my Drupal articles and tutorials or come check out the Coulee Region
Limits are part of the system
Actually, limits are an essential part of the system. It really is related to game theory. In practice, if you're an active member of the community, you'll have plenty of cheers/props to give out.
And if you're not that active, that's OK too ... because your props fill up over time, too.
I encourage folks to create an account on 43Things / 43Places and try it out.
.
Ah, I think I missed that you got some over time even if you're not active. As long as you get some to start out with and can get more even if you're the quiet sort, I guess that's ok.
Michelle
See my Drupal articles and tutorials or come check out the Coulee Region
I commented on this at Leisa's site
Basically, a "cheers" based system like 43things uses. It's a "positive reinforcement only" system, it can be applied to any object (modules, users, posts, issues, comments, etc.), and you either only have a limited amount of cheers to give, or doing actions gives you more cheers.
Please have a look at 43things to see what I mean, and read the comment thread on Leisa's blog post here: http://www.disambiguity.com/social-literacy/
Peers rate content, not wizards rate users
Regarding directly rating users, I think the "positive" only idea is OK, but actually I think we could dispense completely with people rating people directly. (Exception - site maintainers re-setting karma points when they see someone's obviously gaming the system.)
A user's karma should derive indirectly from the community rating the quality of my contributed content.
Also, I don't like the simplicity of up/down, although it's certainly better than nothing.
There are multiple factors, e.g. rating the topic as opposed to the content: the oft-expressed "this should be core" really means "I consider this feature/issue to be important to basic functionality". The negative idea "this is an obscure issue I can't imagine it's relevance". Another is rating the accuracy of content, we need ways to have users flag nodes as needing review, specifying the general problem - out of date, version-applicability is ambiguous, improperly tagged or placed in the nav hierarchy, flat out wrong, etc.
I think something like Flag module might be a good way to go, would also enable per-user bookmarking, perhaps "I have seen and absorbed this and don't need to come back" vs "I want to re-visit this one day" vs "Review soon!"
Of course people can use their own off-site tagging systems for this, but IMO it's valuable meta data d.o. should be capturing and using to bubble the important stuff to the top of the views.
contributions aren't only content
It's hard to rate irc help, commit messages (unless we had flag integration with cvs module, heh), useful posts on drupal planet and a bunch of other things that get done on Drupal.org - so if we had a system like this, it'd need to be able to recognise those contributions rather than simply content as such. Adding flags for handbook pages (and issues etc.) is a good idea, but a little bit out of scope for karma as such.