OK. So I know this is a big hairy, scary one, but I think it needs tackling.
We need a way to better recognise a wider scope of contributions to Drupal for 'social' reasons (so people who don't actually know each other can more easily determine who is who and how they contribute to the project) so that people who don't actually 'commit' anything but contribute a lot have their contributions more recognisable, and then so that we can potentially match people to projects/issues when we need people to help!
We could go crazy with all kinds of complicated reputation systems but I'm proposing that we keep it pretty simple and human driven (I think this is a problem that is better solved with people than technology).
Here's what I'm thinking (and I'll try to post some sketches of this shortly too).
we develop a taxonomy that lists our areas of expertise which could be visual design, theming, user experience, tech writing/documentation, coding, testing, reviewing, pls feel free to add/edit/etc.
we map that taxonomy to 'phases' of a project (ref: the wireframe that I did for the issue template and the different tabs/phases that a project goes through http://bit.ly/hzLsZh ).
terms from the taxonomy can then be associated with your profile on Drupal in one of 2 or three ways. Firstly - you can say that you have expertise in one or more areas of practice. So, I might say that I have expertise in UX for example. Secondly, someone else can look at your project an say that you have expertise in a particular area of practice. So, I might look at your profile and say that you're good at documentation. If I do that, then it will say on your profile that I think you're good at documentation, so this is more than a 'quantifying' tool - it's not necessarily about how many people think you're ace at documentation, it's more about who think's your good at documentation. This will have relevance either if I already know someone who is vouching for you and I trust their judgement OR if I know people who recognise good documentation when they see it and I see they are vouching for you. (These are obviously not mutually exclusive). This means that we will hopefully be more thoughtful about who we vouch for because we're kind of putting our own reputation on the line by doing this. (You don't want to be seen to be vouching for someone who is incompetent or a douchebag, right?) I'm drawing a little on Quora again thinking about how they show who has +1ed answers on Quora not just how many people have +1ed.
terms/areas of practice can then be given more or less prominence by people in the community +1-ing you for that area of expertise. So, for example, I'd go +1 the people who I know who already contribute a lot to UX in Drupal on their profiles, assuming they'd already 'tagged' themselves (eh, I know, I'm mixing tags and taxonomy... this is why it's not good practice to describe implementation as you're describing your design idea!). (See above re: trying to make it not about numbers but more about who is vouching for you).
It's as simple as that. I did think about mapping actual activity to 'terms' but then thought how do you distinguish between good activity and well, less productive activity. It gets complicated fast.
I think what I'm describing above is a great way to take a first step into this territory and then we can gradually tune/experiment with the system over time to see how we can add to it without turning it into a game.
The side effect of this is that we can then use this mapping to put a call out to specific people when we need them to help on a project with out having to know who they are and hope they're in IRC. It also gives newbies and easier way to start getting involved in contribution.
We'll also need to look throughout the site to see where we need to remind/prompt people to acknowledge expertise and find a way to do this so that it's as easy as possible.
(Edited to add notes re: showing who vouches for you and making this more prominent than how many ppl have vouched for you. So it's not anonymous +1s and less gameable/quantifying)