I've been toying with some questions surrounding our perpetual difficulty with docs, and our tendency to burn through docs contributors (perceived... maybe not actual).
Question: Why is it possible to have at least two or three books published, perhaps more, that blow away anything we have on d.o in their clarity, scope, and accuracy? I'm thinking of Pro Drupal Development, for example, which has a clearer statement of what's going on in Drupal code than anywhere else. Why couldn't we have a doc of that quality in our repertoire? The subversion project does? The git project has two. jQuery has two websites that take your breath away in their conciseness and clarity (and interactivity).
My theory is that it may because we focus on only community documentation, which by its nature gets fragmented. People just scratch their little itch and it all gets out of hand so quickly.
Could we create a comprehensive, high-quality tutorial or reference like git or subversion or jquery have, and maintain it? How would we do that? Would it break our community culture? Could it be properly maintained? Why is the absolute best documentation being done outside our community, in the book publishing world?
Looking forward to your answers.
Comments
I'm not sure how
I'm not sure how git/subversion/jquery et al do it, but one of the major differences with published books is simple QA. In other words, they generally employ professional editors (sometimes different technical, style and copy editors plus a dedicated indexer) and usually some kind of formal technical review. And the editing and reviews can be ruthless. Stuff that's even slightly vague or confusing MUST be rewritten. Anything out of scope gets chopped. It also helps that writers are working to some kind of carefully considered plan (usually informed by the publisher's experience with similar content). At least that's how it's supposed to work. From reading the reviews on Amazon where people are always complaining about buggy code examples etc it seems like the system doesn't always work that well.
The Drupal documentation on the other hand is almost all contributed on an ad hoc basis with very little planning and no formal reviews (editorial or technical). The big upside has been that we've amassed a huge repository of generally useful information. The downside is that there is also a fair bit of ambiguity, duplication, misinformation, rambling, inconsistency etc.
I don't think the formal publishing model is ever going to make sense for Drupal.org. I think we'll always want people to follow the wiki model and just make contributions in a totally frictionless environment. But I think huge gains are possible by encouraging the gardening process: removing weeds, trimming the overgrown bits, reorganising etc. and by involving more people in technical reviews, especially of the more critical chunks of the docs.
Maps and itches
I agree with LeeHunter that a pure publishing model wouldn't work. Or perhaps, more correct to say it would trade one set of problems for a new set of problems that might be even thornier! I do think it's worthwhile to explore for ideas, though.
Part of the problem for me is the lack of an over all documentation map. Books are a good example of writing to a plan. I've thought for a long time that someone with a good sense of the whole needs to lay out a table of contents or documentation map so people can come along and fill in the sections. This isn't a magic bullet, I can point to places where this has been done on the small with meager results. However, having a map/TOC in place would, I believe, lower the barrier. Unfortunately, like most architecting, creating one requires a specific skill set and depth of understanding difficult to find.
There's an emotional aspect to this, as well. With a map, you know where you are and how far you are till you're done. There's a greater sense of accomplishment. Yes, you're never really done, there's always maintenance. But milestones are important. It sucks investing gobs of time and walking away not knowing if it made any real difference.
Regarding "itches". A book puts deadlines and rewards in place for working through the hard and tedious bits. Scratching an itch is another way of focusing through hard and tedious bits, and giving someone a comfort level about their understanding. Without such a focus, you have to arbitrarily learn something in order to write about it, and even then don't have the same assurance you've got it right. I had some handbook stuff requiring feedback because I had newly learned it. By the time I got it, a new project had consumed my time and I never did get back to updating the topic.
The one vs. the many
As a published author of how-to books, I concur that the sequential structure of well written books is hard to accomplish in a community setting. Most books are outlined and written by a single person who can build a good presentation structure up front. Book editors then help to reinforce and improve that structure. "Documentation by committee" will always result in a more chaotic and messy approach. Beginners need structure and a course outline. Intermediates need quick access to answers. The two needs are hard to merge.
Part of the problem lies in the limitations of Drupal's book module. Handbook structure is more a result of "when" contributors add certain content. We need a much easier way to restructure and merge content, and that will have to be done by a small and well coordinated group of editors. The book module is notorious for being hard to organize after the fact.
Here are a few suggestions for improving the community structure: (note, I am a Drupal intermediate, certainly no expert, so many of these suggestions may already exist or, are not possible)
1) The community provides the content. The documentation team provides the structure. This will help to create a more cohesive body of material with some degree of structure to it. The comments on articles are an excellent way for the community at large to help add and revise content, but the documentation committee should periodically review and merge comment information into the main article. There is a feeling that we shouldn't touch other people's comments, but that ends up scattering good information all through the comment section, information that should be placed (and edited) into the main article by the "editors." (The documentation committee)
2) Pictures are worth a thousand words. We need a much easier way for the community to upload images (and perhaps videos) to handbook pages. They should be uploaded to drupal.org rather than linking to outside sources, so that links never go stale. Imagecache could then be used to resize images for theming compatibility. Lightbox could be used to present large screen grabs in the appropriate size for easy reading while not breaking the handbook's theme.
3) Quarantine different Drupal versions. There should be separate and distinct handbooks for each major Drupal version. Mixing documentation for different versions multiplies confusion and the difficulty of finding the right solution for the right problem. If a piece of documentation could apply to multiple versions, then it should be duplicated into the other handbooks. Perhaps cross links could be added at the end of articles to provide a quick path to other versions.
4) Easy hyperlinks. We should make it as easy as possible for a contributor to link to articles elsewhere in the documentation. This should NOT be done via NODE ids because it requires that documentation content be "frozen" once published. We need a way to combine and reorganize nodes as the documentation evolves, which would break node links. Perhaps a tagging or title reference could be used. Wiki's manage to handle these types of links in a very logical and simple fashion. (should we just use a wiki for documentation anyway?)
5) Bring the documentation to the user. The help, or advanced help module within Drupal core should provide a context sensitive link back to the online documentation. For instance, if a developer is on the pathauto configuration page, there could be a link to online help at the top of the page that links to a search result on drupal.org with articles on pathauto. This provides context sensitive answers when needed as a Drupal user is working directly within Drupal. Drupal help evolves as drupal online help evolves.
I realize that some of these ideas will be challenging to implement, but now is the perfect time to consider documentation enhancements, before the "new" separate handbook section goes live on Drupal.org.
Book module could really be a lot better
It's a side topic, but really, the core book module is tremendously limiting. It has so much promise, but it just doesn't fulfill it. I do think that with a bit of work it could be improved. This is an example of a module that would be way farther along if it had stayed in contrib.
Coherence isn't for everybody
Book publishing is great. The Drupal book boom helps everyone. Docs isn't competing with them.
I write and edit docs for d.o., but I'd say I personally find more help using Google, limiting the search to site:drupal.org. That usually takes me to a forum post or issue queue where others have struggled with same problem.
It would be great if more solutions to problems that were scattered around issues and forum posts wound up on docs pages that would be easier to follow than a meandering forum post. But I must say I've gotten very good at reading forum posts/issues and finding the best solution presented, quickly
And we must remember that Google (and I'm sure other search engines as well) makes the problem of "the answers are spread out all over the place" much less of an issue.
I would just love to get more people writing docs. I think the best docs come from someone having just figured out how to do something, having pulled together a bunch of different how-to's, hints, etc and says to him/herself, "The recipe for this should be all in one place." Inspiring more people to stop and take the 45 minutes to write up a success that they've just achieved could provide great improvements in docs.
I also know that others are more into coherent systems of ordered learning. And I support people who want to work on those kinds of efforts; I personally am more interested in the less coherent approach.
Shai Gluskin
Content2zero
Ask other communities
I have struggled with this issue for several years now with the Drupal project, where my biggest pet peeve is the versioning that zoon_unit touched on.
@rfay - have you considered posting to some of the projects you mentioned and asking them how they generated (and maintain) such a high level of documentation through the community?
I think it would be great to post to forums, mailing lists, and directly to any documentation coordinators in those projects to ask about things like work flow, tools, team structure/dynamics, etc. so that we can really try to extract any commonalities between these projects.
I would add that the Python community also seems to have done a great job with docs, and I have heard good things about the Django docs as well.
--Ryan
Ryan Cross
Drupal Development Services
ProjectPier project management and collaboration software
OMG! I just counted the pages
I always knew there was a HUGE amount of content in the Drupal documentation set but I never knew quite how much until today when I downloaded all the handbooks and converted each one to PDF.
Here are the raw page counts:
Does that number (6700 pages) sound scary? Well, it gets worse:
So these numbers underscore a number of issues:
hoo boy. wow. no really. WOW.
hoo boy. wow. no really. WOW. i knew there was a lot, but that just blew my mind a little!