A new dynamic hub for the Drupal 8 Multilingual Initiative

Drupal.org is great to maintain a solid number of issues but all of us trying to wrangle a huge number of issues need way better tools to have overviews of what is going on and how is work structured. We need to be able to help new people join in quick, let them have an overview of current tasks and also on the place of tasks in a bigger picture. So we need to have the bigger picture documented with the possibility for people to dive in on demand and we need to keep it always up to date and relevant.

The Drupal.org issue queue is great to see a status of issues, but it is not good at all in maintaining relations and especially bad at providing support for hierarchy and in-context structural information. All we get is basically a list of issues like this one: http://drupal.org/project/issues/search/drupal?issue_tags=D8MI

Recap of predecessors

Different people got to different approaches to these problems. Here are some previous solutions that Jacine and I looked at and tried and the reasons I think they did not work the way I wanted them to work for me and people working with me.

1. Documentation pages. Jacine did a superb work in the HTML5 initiative collecting tasks into documentation pages with explanation on how can one contribute. One such page can be seen at http://drupal.org/node/1183606. This is great for getting some overview of the tasks at hand, but there is not a lot of metainformation on the issues and it needs manual editing to keep up to date. It cannot be dynamic in itself to show current tasks. Regardless, the documentation tree for the initiative (starting at http://drupal.org/community-initiatives/drupal-core/html5) works well for the people working on it as far as I've heard. I was just looking for a more dynamic solution.

2. Issue trees. I started to make up an issue tree for the Drupal 8 Multilingual Initiative back in August (in preparation of the London code sprint), so there would be an overview of the perts of the initiative. Turned out there are so many levels required that META-issues needed a META-META issue (http://drupal.org/node/1260534). Many people don't know what's a meta issue, let alone meta-meta. Also, again, the hierarchy (up-down) requires manual maintenance, and gradually got outdated as work focused on patches instead of maintaining the whole structure.

2a. Issue tree visualization. To make the issue tree more friendly for people I talked to some of my great Hungarian Drupaler pals who went on and created the Issue tree module to visualize the tree represented by the data from above. See http://drupal.org/project/issue_tree. Unfortunately this did not get much support for deployment on drupal.org and the topic pages efforts are in full force to propose something much bigger and better (http://groups.drupal.org/node/144584) so this has fallen through the cracks.

3. Mindmap. So given the issue tree did not have a chance for deployment and I wanted to provide an overview of the initiative, the initiative mindmap was born (and posted on the META-META issue at http://drupal.org/node/1260534). This was also plagued by the data disconnect. When tasks changed the map needed to be updated. Current tasks were only marked here, not on the issues themselves, so I need to manually keep a list of current issues updated elsewhere. This was/is a more up to date version of the structure compared to the issue tree, but is/was still lot of work to update and lacking features like pulling out current tasks or helping people find tasks that need testing or are easier to do.

Enter the Rocketship!

So in frustration of not having a good overview on all tasks even myself and not being able to sleep due to that frustration, the ambitiously labeled "codename rocketship" was born. Experience for yourself at http://hojtsy.hu/d8mi.

Basically rocketship comes from the idea that we need structure and higher level organization for huge sets of issues while at the same time be able to have an overview of both metadata on the issues and current state. It is a combination of all things I liked about the HTML5 book solution with a lot more automation based on metadata exclusively pulled from drupal.org. The initiative hub consist of a hierarchic book at its core, where each book page can have a taxonomy term name specified. If a term is specified, the issue nodes tagged with that term are pulled and sorted into 6 groups based on their status. For the first version I lumped "active" and "needs work" for example under a "To do" label, and did some more complex mangling to sort backport issues into a group as well (i.e. issues that were worked on for Drupal 8 before but are being worked on for Drupal 7 or 6 now).

Pulling issues based on taxonomy terms lets us create a page for example, that does the same sorted overview for current focus issues (D8MI and sprint tag at once, again an idea lifted from Jacine): http://hojtsy.hu/d8mi/current-top-priority-tasks

Then based on "lower level" tags we can dive into only the issues that deal with converting to "langcode" from "language" in schemas and APIs at http://hojtsy.hu/d8mi/migrate-schemas-and-apis-langcode for example.

We can also tag things for different areas, so they show up in different areas of the structure tree in their respective place.

Then when looking at let's say the base language issues (http://hojtsy.hu/d8mi/base-language-features), you can see I color coded the critical and major issues as well as the current focus issues. I also took liberty to mark the "Needs tests" tag with a distinctive color, so people looking to be able to help with that can spot those easily. I can imagine a couple other key tags can get colors, but we should got go absolutely overboard in that. An issue board with rainbow effects is not really useful.

Technology background

The feature works with a local cache of all issues tagged with D8MI with all the information scraped from Drupal.org (given no API available) and stored in local nodes. While that is not a great long term solution, I'm focused on improving Drupal 8's multilingual capabilities, so I tried to get to good results fast. The local cache of issues is updated every hour, so drupal.org should not be hammered at all. Similar issue status overviews are likely possible on drupal.org, I'd love to work on that given resources, however that is probably not going to happen with me personally for another 1.5 years or so given my focus on Drupal 8.