This thread is to allow us to discuss the material on the companion wiki page, what insights we can draw from it, and how it should inform the development of a policy.
Based on the items in the "Suggestions" list, one idea I had was that instead of saying, "you need to do all of these things to be listed in the directory", we instead say, "Here's a list of a number of things you can do to be listed in the directory. If you do X number of them, then you're eligible for consideration."
Another idea is that listings should be periodically reviewed, and the requirements structured in such a way that continuing involvement is necessary to remain in the directory. Roshan did have a valid point in the thread that some people come into the community, make a splash, and then disappear or get tired of/fed up with Drupal for some reason.

Comments
no specific criteria
If we give a specific criteria like "If you do X number of them.." people will do exactly that number in the least painful way just to get listed.
In my opinion, the point of that particular listing is that you not only contribute to the community but you understand the social values of the community enough to "get" the requirements necessary to be listed. Making it a clear, objective system for inclusion removes that. It's the "nose test" - I can't state a set of rules for inclusion but I know it when I see it. I actually like the "WOW" factor that chx and Heine have been using. Companies included in that list should make us think "wow" when we think about their contributions to the community. That's actually pretty easy to test, IMO.
I think the idea of reviewing a company for removal is entirely possible. We've removed hosting companies from the hosting area when they stopped offering hosting.
--
Growing Venture Solutions | Drupal Dashboard | Learn more about Drupal - buy a Drupal Book
knaddison blog | Morris Animal Foundation
Okay, what about this
Okay, what about this idea:
We have a standing subcommittee of folks from the webmaster's team who meet periodically to review service directory applications and decide which ones (if any) had made "extraordinary contributions" to the Drupal community and were worthy of inclusion. This same team would also be able to review existing directory listings and drop folks if they were no longer actively making contributions.
This structures the directory as more of an award or honor for good community members, which it seems is what the community wants this section of the site to be. The problem that I see right now is that it isn't clear to everyone, and the ad hoc nature of how service directory applications are reviewed and promoted just causes more confusion for those individuals, especially as standards for inclusion have evolved over the years.
I guess my big concern, similar to what I was hearing from Amazon as well in the post that kicked all this off, is that right now we're treating service directory applications as "issues", and that model is pretty confusing for businesses that follow all the directions listed on the index page and then are told that they aren't "wowing" the site maintainers enough to make the cut. It's gotta be especially frustrating when they ask what more they need to do, and don't get a clear answer.
Structuring directory inclusion as more of an "award for good service", and having a process where applications are periodically considered by an "award committee" might clear up some of that confusion.
But that's just an idea ;-)
Award == nominations
Then create a page where people can nominate others or themselves. Not webmaster issues. Make it so that there is a minimal word count on the page so people wont use it instead of a five-star widget.
Structure
I grow more and more uncomfortable with the "I know it when I see it" sort of structure. That is ripe for abuse, both on the management side (vis, webmasters) and on the submitter side. Having a structure you can point to is a very important CYA. Even if that stated policy is "we reserve the right to approve people however the hell we want and make no claim of being fair", that is still far far better and more defensible than claiming to be fair but having no actual definition of what fair means.
I like gdemet's review committee; they can and should have a stated definition of how they make a decision, and a statement that their decisions are final, etc. The main goal here as I see it is to protect the site admins from frivolous accusations. A clear structure and policy to point to and divert blame to is a common and effective way of doing that.
ok, write some objective metrics
Great. The folks who like this idea of clear metrics need to start writing them so we can see if they will work or not.
I'm not convinced they will work, but am willing to explore the idea.
--
Growing Venture Solutions | Drupal Dashboard | Learn more about Drupal - buy a Drupal Book
knaddison blog | Morris Animal Foundation
While the discussion both
While the discussion both here and in the webmaster queue has convinced me that using specific metrics probably isn't the best approach, I'd be more than happy to write up some material that would outline the structure/role of a review committee. The last week has been fairly busy for me, but hopefully I can get to it in the next few days.
Yeah, I wrote that back in
Yeah, I wrote that back in July, didn't I? Rest assured that I haven't forgotten about this, but if anyone else has any ideas, feel free to post theme here or let me know!
Can we revive this in Chicago?
I'd really like to see some movement on this, especially as we move closer to a full-fledged Marketplace. Can we plan on meeting about this over a BoF at DCC? (I'd also like to talk about the policies for adding folks to the Planet, but I suspect that's a more volatile subject, and best left to its own BoF)
I'd also like to start to reach out to other people to help out with the directory, as it currently seems like 4-6 people (max) review these, and thus they sit from weeks-to-months without any action. Given how important this listing can be (or at least seem) to new and/or small contributing firms, I'd really like to see some more action here. We need to grow the ecosystem, and this is a relatively easy way to promote that growth.
I'd be happy to recruit more people to help, though I don't have any authority to do so other than my own inflated sense of entitlement. ;-)
Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology
things move quickly when the case is clear
I'm not sure things do move too slowly. When the case is clear, things move quickly: http://drupal.org/node/1023368
I agree that a BOF is a good idea.
knaddison blog | Morris Animal Foundation
Unmoderated version
This is slightly OT but related. I started this unmoderated wiki page a while back and it's gotten huge: http://groups.drupal.org/available-for-hire . While it's great to have a listing that is selective to highlight those who are doing good for the community, we also really need a more structured place where people can list themselves without any restrictions. Maybe they could be rolled into the same place with some sort of badge or other distinction for the "premium" listings?
Michelle
What are the standards for?
As we consider what criteria to use for the services directory, I think it's important to keep in mind the goals of the policy. Here's a great example of why I think it's important to maintain a "contributing company" policy for the listings: http://drupal.org/node/733528 .
Here we have a company that was taking part in and sponsoring cons/camps, but was not contributing their code on Drupal.org. Because of this little "nudge" we were able to get them to move this code to d.o. and it seems like they're now sold on the importance of working within the context of d.o. They haven't been added yet, and I don't think they're "there" yet, but they're sure moving in the right direction.
I think it's generally agreed that we need more contributing Drupal companies, and I think the goal of the services listing should be to encourage companies that are developing their sites with Drupal to contribute to and engage with the community. This also means that our "nos" (which we should try and pose at "not yets") should be as friendly as our "yeses", and it also means explicitly telling people what we expect them to do to get listed. I know there are concerns about gaming the system, but I think we run much more risk of making the system seem rigged and unfair than we do wrt gaming.
Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology