One thing that's been on my mind ever since the inception of the Design Initiative is some form of reputation system for design.drupal.org. If we can make this fly I'd be hugely in favor of it, and to that end I've put together a basic model of what this could look like (including an implementation), I'll get to that shortly, first I want to look briefly at what exists in Drupal ecosystem already - well, at least what I know of.
Lists of people who do stuff
In various dark corners of d.o you can find these lists of people who do stuff, well, are allowed to do stuff - which doesn't necessarily mean they do anything at all. Lists are not useful as a gauge of reputation, nice for seeing who is working on something, or who has a particular role, but apart from that not very useful. Some examples:
http://drupal.org/documentation-team
http://drupal.org/site-maintainers
http://drupalcode.org/project/drupal.git/blob_plain/refs/heads/7.x:/MAIN...
Certified to Rock
Now we're getting somewhere - a tool that actually gives each user a rating. In the complete absence of anything else this has provided us Drupalers with a way of saying - look, I did a lot of stuff and I engaged in the community. There's a good rundown here: http://certifiedtorock.com/about-certified-to-rock-for-drupal - the bit I like the most is this statement "Not only is that a good proxy for knowledge of Drupal, but it also is a gauge of how successful the person is at engaging with the Drupal project". Check out my CTR rating:
http://certifiedtorock.com/u/61393
DrupalModules.com
This site uses multiple vector five star ratings to rate modules - an by proxy module developers. The system is described here: http://drupalmodules.com/module-rating-guidelines, which is all fine and dandy if you build modules, alas I don't.
Ye ole reputation by visibility
OK, so I didn't really know what to name this - but its entirely possible to build a formidable reputation in Drupal just by being here a long time and doing a lot of stuff - in other words people just get to know your user name, see it everywhere and you get kudos for that. It certainly helps of you have some popular contrib projects and work on core - in other words you are probably a developer, but not always, although it is rare for a non-developer to conjure this type of rep.
A basic Idea for Design.Drupal.org
The way this could work is for users to earn points and badges for doing stuff. For example if you submit a node you earn some points, if you submit a comment you earn some points. Site moderators would earn points when they moderate something (as a reward for their volunteer work). We could also allow users to vote up/down on certain content types - so you could earn points from votes (could be gamed, have to careful here). What I like about the contribution + voting points is that its not only a measure of your productivity, but the perceived quality of your contributions. If you produce a lot of crap, well your rep is going to suffer. When you reach a certain number of points you get awarded a badge.
I think this could work, as long as its set it up properly and we can prevent gaming + moderate to some extent the contributions. We would need to strike the right balance of points awarded for contributions (creating nodes, comments etc) and those earned/subtracted from up/down votes.
Now, I'm not saying this is going to happen for design.drupal.org - there's a long way to go with regards to building this site and getting modules and features approved, however I think its worth discussing with designers (that's you) what you think makes for a good measure of reputation and how we might implement such a thing.
Comments
Echoing your words, the big
Echoing your words, the big problem with implementing something like this is the gaming that could take place. Anything in the sphere of voting is open to manipulation from those with the largest following, and once it's taken place it's damn hard to reverse/police.
I think the defining factor in how to create a reputation system must be based on the motivational reasons for having one in the first place.
If the main motivation is to encourage a higher quality of submissions then the system should reward "quality over quantity", ergo a designer could contribute a small number of really fantastic designs and gain a status boost for that fact that what they contribute is really good.
However, if the motivation is to get a fast moving active community then quantity comes over quality, maybe with the emphasis placed on a more iterative process and that the contributor acts more like a moderator of his/her submissions - driving the process.
I personally like the Certified to Rock system, but the problem I see with that is that it's not quality that is measured.. and one could assume that a heavy participator is necessarily a good one. This sort of system works when you need a series of measures and it can build a general profile of an individual. I don't really know anything about the way rankings are made up (apart from what they tell) but I would assume it counts D.O. activity... which could possibly include writing sub on an issue... which is valid but not a contribution.
Where design is the prime focal point, there would need to be a really dedicated obvious difference between "good" and "great" rankings, and for me, I would say it has to be based on the quality of the designs as opposed to quantity of submissions. A "best answer" for discussions/content items could work well for a gauge of community participation, but again it's how to promote quality over quantity.
I like the badges idea, that's a good visual pointer to different peoples strengths. ie. a moderator/admin shouldn't be ranked higher than a quality designer in the same category.
Dreamleaf Media
+1
+1 to dreamleaf's sentiments. I agree, the design world has a whole lot different planetary (eco)system.
Marckee for Short
You raise the essential
You raise the essential problem - how to differentiate the "good design" from the average, or even the bad. I wonder if a vote up/down system would create a self regulating system, so if people try to game the system the community will recognize it and start voting down crappy submissions?
If we do have a reputation system then it has to account for good vs great design, that's the issue I am focusing on solving.
Personally I dont think
Personally I dont think you'll solve the good vs great design problem as you can't control peoples opinions or perceptions towards a piece of work. Obviously you don't want people to upload random shite they created using Artiseer or freebies from other sites claiming it's their own work but I think the measurement between good and great isn't something that can be controlled by a system which leaves that up to the users.
The vote up/down or like system is the most likely candidate for a self regulating system so works are judged by viewer/user participation rather than points added per upload/contrib or creation of a comment for example. To use a current system as an example which I know does have it's issues which is more of an industry thing, dribbble functions as well as this kind of system can without getting to involved in policing things or the admins needing to be too involved with regulating uploadds. The "voting" is done by 2 parameters 1.Followers, users that like someones style or submissions/uploads and can follow said artist and 2.Likes, there's only the option to "like" something or to remove your "like". It works pretty well as far as I've seen, the only downside to the system is that a percentage of the most popular designers are now just auto"liked" by their followers because of who they are and not what they have uploaded.
The question I'd rather think about is what is the outcome this needs to have and who's supposed to be benefiting from this the community or the individual or both? I would of thought the point of the rep system would be to have design work as an advertisement for the designer, a way for prospective clients/developers to see who makes quality work and lastly to show who is in it for the community.
...
Good points Andy.
Have you heard of something called the "ladder of participation"? Likes, follows etc for-fill a similar role - they are low risk ways of entering a community, they are easy (don't take much time) and give the user a sense of being part of something. So on this front we're talking about the individual and the community benefiting. Users can get involved early, and easily. The community gets feedback and gains a new member, who grows in confidence.
Now, having the design work as the prima facie advertisement is fine, if anyone can find it. How will those great works, or those outstanding designers bubble up through the milieu?
I have been testing with the Radioactivity module used here on g.d.o - I think this has promise also, you set a decay rate so content that was popular slowly fades away, while new interesting content springs into the top lists - things like page views and votes count towards a items "hotness". This sort of system would benefit people who contribute regularly and have stuff that others find interesting.
That sounds like an ideal
That sounds like an ideal system, i'd want to make sure that all new submissions get seen though so some kind of view with a limit/balance on how many hot/popular items are shown so that new content is seen and not quickly pushed to one side because of a popular item(s). At the best of times actually having the stomach to submit something which will get some form of critique is a big enough challenge for designers so if popular content takes over that may cause some bad vibes. I know that's going to be more of an implementation topic but could this be an opportunity to think about how designs are filtered or is that for another thread?
I also like the idea Marco suggested with having a "pick of the day" but think that would probably work better per week or month depending on how popular this is used. Only problem with it is, Is that adding too much content to baffle people with?
To game or not to game
The worries about gaming are sure valid.
Personally I am using two Sites that use rating and reviews a lot: Amazon and the Android market.
While the rating in the Android market is crap, the user reviews, when viewed in large numbers, give a fairly solid picture of what the app is like. In general also the reviews are often of low quality and born out of spontaneous frustraiton or enthusiasm.
Different on Amazon: There I feel the system works very well. Maybe even half of the reviews may be false ones (I'd bet for a lower number) still you get a very good picture of the product you are buying. If there is any problem with it, you can be sure someone wrote about it.
But in general I have the experience these review systems, even if not the best one, work very very well and almost always give a realistic picture.
So how about voting is only possible if you write a short review that has to be of a certain length, say minimum 300 Characters? This may keep people away that just wanna click some up- down buttons and forces everyone to give reasons why they like / don't like something.
And above all: If there is doubt about the community consensus and suspect of gaming, I think the people that are in charge of the initiative are free to overrule a voting, if there are strong reasons. This is not a political vote where you automatically get into parliament if you got the most votes.
Sorry if I am a bit off topic, but these are my thoughts about voting and gaming I wanted to write down somewhere for a long time :P
Life is a journey, not a destination
For sure you are right about
For sure you are right about Android market, the reviews do help a lot, but only if you really dig in and read a lot, some are knee jerks like you say. I suppose for our site comments are the reviews, so in that regard there is a defacto reputation system working strait away.
The only reason I don't particularly like the idea of forcing uses to say something before they can vote is that it takes away a "low barrier to entry" (to the community), i.e. the low rung on the ladder of participation. Maybe users don't feel confident enough to say something, but they can safely, and anonymously, vote/like/follow something without exposing themselves.
shirtless
Just as an example, I like the idea of how the tee(shirt) crows-sourcing space deal with creativity, notoriety and reputation.
EXHIBIT A: www.designbyhumans.com
EXHIBIT B: www.threadless.com
Merits go to the design/ers that get the most #-of-votes (to critical acclaim!). After which, the designer feels so loved & cherished knowing that his/her design is being worn by someone else he/she hasn't met.
People who love or hate the design can always chip in their comments which is forever immortalized in cyberspace. For the mere spectator, having a say in something (or someone) gives an opportunity to be interactive -and in effect- improving someone else's reputation.
Marckee for Short
...
Are they really the same thing Marc, most of that looks like voting for a tshirt design - as in a contest, rather than a reputation system? Honestly I haven't looked that closely at those sites, but I will make a point to dig deeper.
...
Sure, in clear cut cases of spamming any site maintainer can step in and delete content or even block users, but vetoing votes would be dark and muddy waters to wade into. Its not a responsibility I plan to take on, nor would I support this. If the system cannot self regulate then its likely to be a #fail.