Specification for our dream (DITA) documentation tool

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!

In February LeeHunter posted his wish list of features for "an awesome technical communication CMS". I've copied his list and processed the comments in the discussion and some of my own, and added our current status implementing these features and ideas of how any missing features could be implemented in the near future.

I'm very much aware that in the coming months most documentation efforts will probably go into getting the documentation updated for Drupal 7, and it's strategically a bad idea to at the same time change from a freeform format to a structured documentation format, rebuild the infrastructure and do a mayor content update.

Nothing prevents us however from dreaming up the ideal infrastructure, and if I know what the doc team is dreaming about, I might be able to steer our (Pronovix') work on the DITA documentation system into that direction ;)

There might also be certain technologies that could make the documentation upgrade less painful (e.g. using the versioning system mentioned below).

That is why this is a wiki, so please add your suggestions! There is also a crosspost on our blog at http://www.pronovix.com/blog/proposal-improved-infrastructure-drupals-do...


VERSIONS - Instead of having separate files for every version of Drupal core, it would be way faster and easier to manage if we could have 1 document, with specially marked up sections for the different versions of Drupal. You could still split the display for the documentation in different tabs. This would significantly reduce the work on future core upgrades.

  • Ready: this is part of the DITA standard, displaying it would be a matter of using different XSLT's. We (Pronovix) have built a UI for this in Spezzle before using RDFa markup in the semantic filters and layers.

IMPORT TOOL - Ability to import XML-based content in DITA, DocBook, or custom formats. Able to monitor file folders or external databases/repositories for changed content and synch automatically or on demand.

  • Ready: We can now import content in DITA as individual file upload and by monitoring a file folder using our feeds extension, this could be used to import from a folder on the Drupal site in which you checkout a GIT repository.
  • Future: Using CMIS it should be possible to connect to Sharepoint, Alfresco and other document repositories. For Drupal.org that would however not immediately useful.

AUTHORING TOOL - WYSIWYG interface for creating and editing schema-compliant content. The writer can easily apply tags to content (but only the permitted tags for a given context!). The author can be given suggestions and prompts. There is a mechanism for conditional text (i.e. the writer can tag text and images so that different versions can appear in different publications). All this would require a lot of JQuery love to be usable.

  • Ready: With poorman's DITA users can create valid DITA without getting lost in the markup options.
  • Selective enrichment WYSIWYG: We would probably want to have some basic WYSIWYG editor that lets users add a subset of the attributes and sub-elements of the main elements inside the CCK fields. A full fledged WYSIWYG editor that is capable of implementing the full DITA spec and that is still easy to use for beginners would be an epic project.
  • full WYSIWYG: My colleague Yorirou has an idea for a technically achievable editor that would be able to deal with any type of XML specifications. Suggestion (from Cliff): Before you get too far into developing this idea, check out XStandard — and specifically XStandard Lite — as a possible WYSIWYG editor. It's configured to be easy to use but also to steer even newbies toward the basic principle of putting structure in the xml or html and design in the CSS. I particularly like its method for dealing with illustrations: As you enter the alt text, you see what the page would look like if the alt text had to be used in place of the image. This should lead to people understanding the role of alt text better, and as a result they should end up entering more appropriate alt text. In fact, I would like XStandard Lite to be available as a WYSIWYG editor for authors and other content contributors to use on any Drupal sites I develop.

PUBLISHING/EXPORT TOOL - Provides drag-and-drop interface for organizing content into single sourced publications (i.e. the same chunk can appear in many places). The publications can be web-based, PDF, XML, online Help etc. Different versions can be built using tags and conditional text.

  • Ready: We've made a system that transforms the .mm format mindmaps from Graphmind into DITA maps (XML), that are in DITA the basis for making documentation from the "single source" topics. We've also implemented an export functionality that uses the DITA Open Toolkit to export to PDF, XHTML, archive, etc. With this system it's possible for users to create their own "Complete guide to building an xyz site" outlines
  • Future: Right now the mindmaps are not validated in the UI against their DITA map convertability (it's possible to create mindmaps that will fail conversion). It would be nice to have input validation in the UI. There are some other modifications to Graphmind that would make it easier to manage documentation (e.g. create new documentation topics from Graphmind, being able to edit topics in the map, etc.)

DOCUMENTATION CLIENT & SERVER - Let's you use and update documentation from inside a Drupal installation context. This feature would make it easier to bridge the gap to end-user documentation in individual projects. This could do for the documentation team what the localization server and client did for the translations teams worldwide.

  • Proof of concept: We've got a first proof of concept for a documentation server that lets you connect specific form elements in your installation with help topics on the documentation server.
  • Future: Advanced help and Help Inject make it fairly easy to package and reference help topics with html files that can be shipped with an installation. In the future it would be great to export all the help topics for a specific project into a help feature that than makes these topics available in their respective contexts.

MEDIA INTEGRATION - As the thecontentwrangler mentions in his comment video and interactive media are very important (e.g. heather's presentation in Drupalcon communicating drupal visually).

  • Ready: Technically this is solved, we would need to decide what content we would want to agregate from where and how.

REUSING EXISTING DRUPAL CONTENT - As Andyoram mentions in his comment there is a lot of documentation snippets out there spread across all the Drupal blogs that are currently only searchable with Google. We should be able to map these snippets from the documentation site and possible reuse them in more extensive documentation topics.

  • Ready: with prepopulate it's easy to make a bookmarklet that would let users submit these snippets to their documentation trail on the documentation server.
  • Future: We've previously developed faceted insert, a proof of concept for a search plugin that let's you search for nodes on a site using faceted search and than insert the found text into a text area. Right now that works with faceted search, but it should be not too complex to make this pluggable and use snippets from Solr.

INTEGRATING WITH API.DRUPAL.ORG - Currently the API documentation and the docs on d.o. are not really integrated (except for individual in content references) It would be nice to more closely integrate both systems.

  • Future: Ideally API documentation would be available as reference DITA topics, maybe there is a way to automatically map the docs to DITA references?

WORKFLOW - We would need to figure out what workflow/permission set will get us the best quality/number of contributions trade-off.

  • Ready: Technologically this should be an issue, Drupal contrib has everything we need for this. This is more an organizational issue

SCHEMA - We could make a DITA specialization for Drupal e.g. have a topic type for modules, themes, features and installation profiles.

  • Ready: Technically this is not hard, we can use CCK to build a couple of extra poorman's DITA topic types specifically for these purposes and extend the module to transform the form into valid DITA. We however would need to figure out the types of information we would like to include in these forms.

TAXONOMY - With RDFa in Drupal 7, keywords be will marked up in RDFa and sites will become queriable with SPARQL. Now would be a great time to make a central Drupal vocabulary for blogs that write about Drupal, so that the Drupal blogosphere becomes queriable as a whole.

  • Ready: Neologism which was developed by DERI is built for this purpose. If we can get this tested and documented, we could have several Drupal shops use the vocabulary once they switch to D7.

FLAGGING - As jhodgdon mentions it would be great if users could flag content in the documentation with flags like "to look at later" or "daily reference".

  • Ready: Easy win with the flag module

Comments

Taking a step back

arianek's picture

Hey, trying to move specifics of the discussion over here from the other huge thread. I'm not going to edit the wiki just now, but want to try and understand how some of these tech specs would theoretically relate to the more general needs and methods of the Documentation/Docs Team. (Forgive me if any of this are things you have discussed in the past with Addi, and for the major holes in my understanding of DITA, I'm just starting to get up to speed re-reading everything.)

As mentioned (http://groups.drupal.org/node/109089) we want to spend some of time at this docs meeting in December discussing overarching (technical) needs and potential solutions as a precursor to spending some more time evaluating options. (Evaluation in earnest will probably be focused on more early-mid next year, and possibly by a dedicated group of people, preferably after the most critical Drupal 7 Docs are dealt with. Which reminds me, it would be amazing if anyone can dedicate a little of this fantastic energy towards helping with that in the meantime, so we can get there faster! Hint, hint...) ;)

After reading through the various DITA threads, watching the Dojo video, etc. I have some questions (for Kristof or anyone else to give input on and discuss) that maybe some of you can help me with, that I think will help us evaluate where this fits in as potential options for tools we may want to implement down the road on Drupal.org:

  1. Is this now employing RDF *instead* of XML? Can you explain a little more about the (dis)advantages of each?
  2. How does DITA specifically help to enable more Docs Team members to contribute more easily?
  3. What is your opinion on implementing the "structure" of DITA before implementing the actual system in the backend? (This question comes from something Addi mentioned that I don't fully understand.)
  4. What kind of effort would realistically be involved in performing the migration of the docs on Drupal.org into such a system?
  5. Would all existing Documentation have to be reformatted?
  6. What amount of resources would be needed to implement and migrate and reformat? (Best guess on hours/# of devs/skill level is all I'm looking for here.)
  7. What would you anticipate being the likely pitfalls, difficulties, or roadblocks we'd run into if we tried to implement something like this on Drupal.org?
  8. What needs would DITA *not* address that we'd still be left with?
  9. Do you see any simpler (but still effective) ways to address some or all of the needs listed above?

Thanks for any feedback/answers/input!

Add Kristoff virtually to discussion at Docs Sprint

rfay's picture

Perhaps we can take some time in the docs sprint for this and meet together virtually.

Possibly

arianek's picture

Maybe right at the start before it gets too late in Europe? I have to work in the morning, so was thinking we'd start the meeting at 1 or 2pm, we hadn't quite settled on a time yet.

Otherwise, I honestly don't think we'll have time to go into huge detail on this, and think we need to focus on reviewing and defining our needs at this point. Then we can move on to actually evaluating and researching options as a next step. So regardless of whether Kristof can skype in or something, I have no doubts we'll be scheduling some kind of more extensive virtual discussion down the road where we can talk about this with him in detail.

Would still like to get some feedback on these things out in the open so that everyone can keep reviewing/discussing either way!

Agreed

rfay's picture

I agree that we aren't prepared to go into great depth, but I think the people who have taken the DITA idea forward with so much intensity deserve as much introductory attention as we can give.

I'm still a believer that technology isn't our biggest issue, but ownership structure of documentation is. However, I love tools that make things better!

We could possibly do something Saturday morning as well, more convenient to European time.

For sure, I just actually

arianek's picture

For sure, I just actually want to make sure we are able to give it some proper/dedicated attention, so even if we do talk briefly, something more involved will be needed down the line... (also, I think I heard somewhere, maybe on the Dojo vid that Kristof is coming to Chicago, so it'll be a good opportunity for some in-person discussions).

It's possible we could try to do that on Sat, maybe if we try first thing before too many others start showing up? Once eager docs writers start turning up though, we'll have to get sprinting.

Media Integration

wfx's picture

With regards to the MEDIA INTEGRATION listed above, there needs to be a way for casual users to be able to report when a linked video is broken. It's great to have free content delivered via Youtube or Vimeo but times change and those services may not always be free. We don't want to have broken video players in our docs any more than we like finding broken images in forum posts because someone's imageshack account isn't up to date. Having an easy way to report problems, maybe even just a little caption below media posts that says "is the video player above broken? If so report the issue, [here]"

Please forgive the newbie

judymcla's picture

I've just spent much of my afternoon poring over modules and other things (like this post) trying to find precisely this functionality:
Provides drag-and-drop interface for organizing content into single sourced publications (i.e. the same chunk can appear in many places).

I see that some aspect of this is ready... somewhere. Can anyone help me out? Is there a module that enables this functionality that's out in the wild somewhere now?

I'm trying to figure out how to redesign the documentation for one of our sites, and this would make my life LOTS easier.

Thanks.

Are you basically looking for

wfx's picture

Are you basically looking for a way to display documentation content (book pages) the same way we control Blocks? I would love to see this as well but I don't know if it exists. Interested to see what others might say, I too hope there's a module I've just overlooked.

What I'd like to do is create

judymcla's picture

What I'd like to do is create a reference guide (book) that explains ALL THE THINGS about the software. And then I would like to create user-geared task-oriented booklets by putting together the relevant pages of the reference guide in a stepwise fashion. Because no one reads reference guides, but people WILL read something that talks about something they want to do.

And I know I can create pages that link all the pieces together, but they won't be books, you know? The user can't click Next and get to the next step.

My (more complicated) ideal would be to be able to mark certain text passages in the reference guide for dynamic import into a book page in a different (user-task-oriented) book.

Mostly, I'm trying to avoid the having-things-in-three-places problem, and address concerns of the project manager and project owner regarding doc usability.

Useful Modules

wfx's picture

Man, this is exactly what I've been contemplating for our documentation. We have a boatload of software to support and it would be nice to reference rather than recreate so as to not have to write things in multiple spots.

I'm still working on this problem but it seems like the Wiki type modules can help some. Here are the modules I've found so far that might help with a documentation/FAQ type site. There's a lot here but maybe it will save you some time looking.

Node Hierarchy - Node Hierarchy allows nodes to be children of other nodes creating a tree-like hierarchy of content. http://drupal.org/project/nodehierarchy

Menuless Node Type - This module allows you to turn off the Menu section in the node creation/edit form per node type. This can be a very nice usability improvement since typically only the "Page" node type is used in the menu tree. http://drupal.org/project/menuless_nodetype

BookMadeSimple Module - Makes books more intuitive. Easy to add child pages
http://drupal.org/project/BookMadeSimple

Book Blocks - The bookblock module can generate an individual menu block for each of your site's books. Also see Advanced Book Block http://drupalmodules.com/module/book-block

Flatten Book Module - http://drupalmodules.com/module/flat-book
(perhaps good for a FAQ as it allows linking to specific pages)

The Flat Book module allows you to "flatten" subsections of a book by displaying them on a single page.
A table of contents block is provided to allow the user to quickly jump to a particular node on the page. Back to top links are also provided below each node. If the user requests to view a node below the maximum menu depth, he will be redirected to the portion of the flattened page that contains the requested node.

This module is useful for organizing large amounts of hierarchical data in a user friendly way. Children of a node in a book are usually related to (talk about the same subject as) the parent. However, if the book tree is deep enough it can become a hassle for the user to drill down all the way to the particular node he is interested in. Furthermore, this flat structure can allow the user to read through entire sections of the book without frequently changing pages.

FAQ Module - The Frequently Asked Questions (faq) module allows users, with appropriate permissions, to create question and answer pairs which they want displayed on the 'faq' page. The 'faq' page is automatically generated from the FAQ nodes configured.
http://drupal.org/project/faq

Wikitools - (freelinking module works with this. )The wikitools module provides some settings to get a more wiki-like behavior. It aims to be lightweight; all features are optional, and it provides no database tables of its own. If you have ideas about cool new features for this module, please post an issue.
http://drupal.org/project/wikitools

Freelinking - To enable interal wikipedia-style links
http://drupal.org/project/Freelinking

Table of Contents Module - provides table of contents. Can be used with wiki pages http://drupal.org/project/tableofcontents

Thanks for the list! I will

judymcla's picture

Thanks for the list! I will sit down and have a look through these modules and see what I think will work.

Issue...

jhodgdon's picture

We have an issue where we're discussing this functionality, and several people that are working on ideas/modules for this have posted code and demos:
http://drupal.org/node/995370

One More

wfx's picture

The good thing about Drupal is that there are a wealth of modules out there.

Outline Designer - http://drupal.org/project/outline_designer

The Outline Designer is a user experience module that makes book management more intuitive. Essentially it overlays on the admin book outline page so that you can use AJAX to build and edit site outlines much faster then Drupal traditionally allows.

Context menu with buttons Add child content, Edit and View Shortcuts, Rename, Duplicate branches of content, Delete branches of content, change node type
Rename content inline
Reorder content via drag/drop with AJAX updating
icons on settings page per content type
Collapse branches for easier viewing as well as set default state of collapse on load
Integration with Organic Groups via helper module (1.2+)
Set theme / skin of content menu (1.2+)
Collapse / Expand All branches (1.2+)
context menu permissions per role (1.3+)
iterative duplication with additional token (1.3+)
permission structure that will allow normal users to outline parts of books (1.3+)
Plays nicely with Book Manager module (1.3+)
Version 1.3

More info like tutorial links if you follow that link.

Problem is...

jhodgdon's picture

The problem with using this (very nice) module is that the Book module only supports one hierarchy -- a page can only be part of one outline/book. For drupal.org online docs, we are hoping for a solution that supports having a single page be part of multiple outlines/books.

Can you elaborate a bit on

valeriod's picture

Can you elaborate a bit on the use case?

It looks strange to me that you really need a hierarchy where multiple chapters point to the same page. I'm wondering if a "related pages" would work.

On the other hand it shouldn't be difficult to change the one-to-many to a many-to-many relationship in the book module.

For instance, if I have a

judymcla's picture

For instance, if I have a reference guide that describes the menu options, as they appear in the menus, for:
- Engine Room Contents
-- Dilithium Crystal Chamber Options
--- Open the Chamber
--- Close the Chamber
--- Activate Chamber Monitoring
--- Generate Crystal Report
--- Release Crystals
--- Synchronize Crystals
-- Warp Core Options
--- Shut Down Warp Core
--- Start Up Warp Core
--- Empty Warp Core
--- Eject Warp Core

What I want to do is then create a User Guide book for common tasks involving these options without duplicating already-existing content:
- Replacing Dilithium Crystals
-- Why Replace the Dilithium Crystals? (new page)
-- Shut Down Warp Core (from ref guide)
-- Open the Chamber (from ref guide)
-- Release Crystals (from ref guide)
-- Close the Chamber (from ref guide)
-- Synchronize Crystals (from ref guide)
-- Troubleshooting Crystal Synchronization (new page)
-- Activate Chamber Monitoring (from ref guide)
-- Start Up Warp Core (from ref guide)
-- Other Important Tasks (new page)

I am just jumping in and

idcm's picture

I am just jumping in and haven't had time to read all that is here so forgive if this is duplicate idea. Based on your use case, I see one challenge - what condition would tell Drupal which previous/next links to show on the node? Does the previous link go to original or the guide?

If you don't want to duplicate content (and who blames you), maybe something along these lines. For each page that is a repeat, create a node without content. Create a node view that shows the node you would have wanted to show. Display the node view in the empty placeholder node. Follow? This is the cryptic how-to. If this interests you but you need the details on how to do it, let me know.

It isn't pretty but it does allow you to maintain the content in one place but have it updated automatically where it is being used in other places. Also, if you are using node aliases, you might need to tweak the title a little or be happy with .com/title-page-0 as the alias.

So it would behave like a

valeriod's picture

So it would behave like a unix symlink; a pointer to something else.

In 3D we can instance objects

wfx's picture

In 3D we can instance objects so you don't have to model reoccurring elements like grass over and over again. You make one blade of grass and then make bunches of virtual copies based on the original. Ideally I'd love to see this done with text but I don't think it's possible.

I guess iframes could work as well to show redundant content but it isn't a practical solution for day-to-day editing.

I could create a node_alias

valeriod's picture

I could create a node_alias module that implements a content type that doesn't have any content. The only field would be a node reference. When you view a node alias it will pull the content from the referenced node.

If you use the "Book made simple" module then you could select a node_alias content type instead of a book_page when creating a child page and reference the node whose content you want to show.

From the 30kft view looks pretty straightforward but of course the devil is in the details (i.e. all the manipulations -- hooks -- that other modules might do).

That's a good idea, I never

wfx's picture

That's a good idea, I never would have thought of that.

Just by accident LOL I'm

valeriod's picture

Just by accident LOL

I'm working on cloning nodes right now so it was an easy association.

I think something like this could have an audience so I'll register a project and start working on it.

I just found this

valeriod's picture

I just found this http://drupal.org/node/18820 but it looks like there has been no follow up.

in that thread jhodgdon

arianek's picture

in that thread jhodgdon pointed to, there is specifically a great prototype in the video in comment http://drupal.org/node/995370#comment-3904984 - johan falk from node one (in sweden) has been working on some solutions to help us for the drupal.org docs infrastructure. definitely worth a look - what he built is really great!

That is great! My brain hurts

wfx's picture

That is great! My brain hurts a little just watching all the switching about in Views. Do you think as this matures that it might turn into a custom distribution like Pressflow?

Heh, I know ;) but it's

arianek's picture

Heh, I know ;) but it's pretty amazing when it's all put together, hey? I'm not sure what plans he's got for it (and it seems that the prototype for the Docs infrastructure is going in some other directions), but he did post a lot of detail of how to replicate that. I bet if you post on the issue and ask, someone will reply - Johan's d.o username is itangalo (if you want to ask him on there).

@arianek I'll message him and

wfx's picture

@arianek
I'll message him and see what he says about making it a custom distro later.

@judymcla
I thought about what we need and templates per subject make the most sense to me. So, as usual when I change my search parameters by one micrometer in Google another module was revealed.

Node Template
http://drupal.org/project/node_template

The function of nodetemplating is very useful for content editors. This module uses the structure and data of a node as a template, it can also define a part of book nodes as a template as well.

Use Cases:
1. You've already created a lot of book pages as a template for your product handbook. Now you can mark the start page as a template, and every time you want to create a new product handbook, just duplicate it with the template!
2. You want to create a lot of pages with minor changes. Usually you must create a node, and switch to the old node to copy its content, then go back the new node and paste everything. Now it's just one click away.

DocBookWiki - A documentation distribution

dashohoxha's picture

I don't know the results of this discussion, whether a distribution about documentation ever got built, but I like this idea. Actually I have created a module that can import, export, create and edit DocBook documents. It extends the functionality of the Book core module. Then I thought that it would be nice to built a distribution about documentation, which integrates all existing modules that extend the functionality of Book module, and maybe develops any new modules if needed. I have just started it:
- https://drupal.org/node/2069447
- https://groups.drupal.org/docbookwiki
If you are still interested on this topic you can join.

However I am not sure whether DITA is something useful, that can really increase the productivity of the authors or the quality of the books. The only way it can be acceptable, in my opinion, is only in the form of node_alias (discussed in the comments above), which displays the content of another book node, and maybe modifies it a little bit on the fly (for example using find/replace patterns).