Index & Hierarchy API: browsable data display based on (virtual) hierarchy.

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Xano's picture

Official proposal at GSoC.

Introduction

Vocabulary Index is an increasingly popular module that allows people to browse taxonomy in a variety of ways. The key element of this module is its simplicity: administrators can set it up and configure it with no more than a few clicks. After the introduction of version 2, I got numerous requests for integration with other modules, like Views and Transliteration, and other data sources than Taxonomy, like CCK.

Proposal

I want to create two new modules. The first is Hierarchy API. This will become a standard API for creating and manipulating hierarchies. Other contribs like Hierarchical Select are likely to make use of this. The second new module will be Index, which will succeed Vocabulary Index. Index will allow site administrators to create pages or blocks where visitors can browse hierarchical content (Taxonomy, Books) or content that has been put in a virtual hierarchy (Users > Nodes or Terms > Nodes, for instance). To do this, Index will depend on Hierarchy API.

  1. Hierarchy API
    1. Create hierarchies based on Drupal data
      Hierarchy API will convert Drupal data to hierarchies in a standard format, usable by Hierarchy API itself or other modules.
    2. Manipulate and traverse hierarchies.
      API functions will allow modules to access, manipulate and traverse the entire hierarchy or single layers or items separately.
    3. API
      Through hooks modules can define which data types they can convert to the standard format used for hierarchy items. Hierarchy API will support basic data types as found in core and CCK, but through the API any kind of data can be inserted into hierarchies.
  2. Index
    1. Virtual hierarchies
      Every index page or block can display a virtual hierarchy. A lot of data is related to eachother, but not necessarily hierarchical by default. Using relationships between data types (Users and nodes are related through the uid field, for instance) virtual hierarchies can be generated and displayed to users.
    2. Views integration
      If Views has been enabled, a Create as view checkbox will be available when creating a new index. When checked, Index will automatically create a view that acts as the index' data source. Using Views UI, people can customise the data result set into detail.
    3. Transliteration integration
      Transliteration is now being done like in Pathauto 1 with a custom transliteration file. In the future we will use Transliteration.module, as it is more flexible than the current custom approach.
    4. Better template support
      Currently, all lists have the same markup. They only differ in that they can have different CSS being applied to them. For more custom theming, we need theme callbacks with wildcards, like Views does.
    5. Theming
      The display modes are different on a technical basis. However, administrators may want to display like like Taxonomy List does, using term images. I want to add extra template files and CSS so administrators can easily theme indexes by copying and pasting those files.
    6. Independence
      Index will be fully functional on its own. However, the presence of modules like Views, CCK and Transliteration will enable extra features. This way, Index' development will not be hindered by other modules, but it will still integrate with them. Example: we don't have to wait for Views for Drupal 7 to create a D7 port of Index.
    7. Interface
      The interface is one of the most important parts of Vocabulary Index. Its simplicity is greatly appreciated and therefore it needs to be maintained, despite a lot of new features being added. I want to add live previews and use AJAX to handle form submission, so creating new indexes will go more fluently.
    8. API
      Hooks will be added to define display modes (browsable, threaded, alphabetical, etc.)
    9. Documentation
      Documentation will be written in the form of Doxygen comment blocks, help and handbook pages. Also, API docs will be made available at api.nederdev.com.

Road map

  1. May 23
    Get familiar with the Views and CCK APIs and discuss what people want exactly. Write down a plan based on the possibilities and people's wishes. Also define the standard format for Hierarchy API's hierarchies.
  2. June 20
    Finish Hierarchy API. Test performance. Write documentation.
  3. July 31
    Finish basic version of Index. Index pages and blocks can be made. Basically everything without Views integration.
  4. August 10
    Views integration.
  5. August 17
    Adding AJAX and bug hunting. Dot the i's regarding the interface. Write documentation. Make it ready for a release.

Communication

I want people to be involved, because this project is something to be used by those people. Everything I do will be posted at g.d.o and my blog at nederdev.com so people can track my progress and comment.

Profit for Drupal

  • Maintainers will have access to a general API for creating and manipulating hierarchies. Modules like Hierarchical Select may profit from this very much.
  • Views will be more powerful with new ways to display data.
  • Drupal will have an easy-to-use system to browse any kind of data. Both for administrators and end users it will be extremely easy to set up and use Index.

Difficulty

Medium/hard.
Hierarchy API needs very good performance. I expect Views integration will be relatively hard.

Mentor

About me

I am Bart Feenstra, known to some as Xano, to others as Kaaskop, although I recently changed that IRC nickname to Xano_ to match my (groups.)drupal.org username. I am a twenty-year-old student at the University of Twente (The Netherlands), where I am a first-year bachelor student in Communication Studies.
Regarding Drupal I am mostly active in the Dutch community, helping people out on the forums and IRC and organising events (DrupalJam, the bi-annual Dutch DrupalCon; Kroegmeet, intended for socialising and expanding your network). Next to that I maintain several modules (Vocabulary Index, External links filter, External Search) and write patches for Drupal 7. I have an excellent understandig of HTML, JS and CSS. My PHP skills are good as is my knowledge of Drupal and the API.

Comments

Great for Ubercart

stephthegeek's picture

Thanks for pointing out this module, I hadn't heard of it before!

Another specific use case that comes to mind is for Ubercart. Currently Ubercart has its own custom catalog display which does not use Views. While you can re-create much of the catalog and product listing functionality with Views to build your own catalog, it lacks the drill-down display from the top of the vocabulary like your module provides. In combination with Taxonomy Image, I believe this module could completely recreate the Ubercart catalog, if it had Views support.

I can see how this module is popular and would love for this project to happen :)

~~~
{ Drupal Themes from TopNotchThemes }

+1

mlncn's picture

This is a good module that I hope will add to a growing trend in Drupal – a module that is easy-to-use and meeting specific needs, but with integration into Drupal's most powerful and flexible systems– Taxonomy and Views, in this case. By making extensions to the general purposes systems so that the make-it-simple approach can be met with these back-ends, everyone wins.

benjamin, Agaric Design Collective

benjamin, agaric

I had a discussion with

Xano's picture

I had a discussion with merlinofchaos and eaton about this and they had a very good way of integrating Vocabulary Index with Views: every index generated by Vocabulary Index is in fact a view, allowing users to use Views UI to customise the index in greater detail, while others could do with the simple UI of Vocabulary Index. While technically this is a great approach, it also makes Vocabulary Index completely dependent on Views. Since Views is such a big beast, porting it to Drupal 7 will take a while, causing a Drupal 7 port of Vocabulary Index to be 'delayed' as well. Therefore I am looking for a way to make both modules work with eachother, without either of them depending on the other.

A solution for this would be to add a 'Create as view' checkbox when creating a new index if Views is enabled. This way we do need custom data storage, but Vocabulary Index doesn't depend on Views anymore but is still able to integrate with it.

Views support without dependancy

kerberos's picture

Yeah, I think that would be an elegant solution to the problem and create a flexible product for everyone. Now if we could just fix the translation of taxonomy terms... :)

-Daniel Chvatik
Adulmec LLC

Term translation in

Xano's picture

Term translation in Vocabulary Index, you mean? That's fixed in the latest dev :)

You're probably talking

Cito's picture

You're probably talking about http://drupal.org/node/372031. However, I found this is insufficient. I have proposed a new patch at http://drupal.org/node/420042.

This project seems to meet a

kleinmp's picture

This project seems to meet a lot of 'official project' criteria:
1. It has student attached to the project.
2. It's well documented
3. The scope seems reasonable for 8 weeks.
4. It's a popular project.

I love the idea of using this module for an Ubercart catalog. Maybe we can think about making this an official project?

Matthew Klein
matt@zivtech.com

Badly Needed!

Alex UA's picture

Very well thought out project, Xano! This would be a huge improvement, that our company has needed multiple times.

Moving to the official ideas list.

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

I updated the proposal with

Xano's picture

I updated the proposal with extra and more detailed information.

I just published the

Xano's picture

I just published the proposal at the GSoC website.

If people have comments or ideas, please share them :)

This proposal now includes

Xano's picture

This proposal now includes the creation of two modules. The second one is Hierarchy API. This module will allow other contribs to create and manipulate hierarchies based on any kind of (Drupal) data. Another candidate for using Hierarchy API is Hierarchical Select.

//Edit: The original post has been updated with more information about Hierarchy API and a more concrete timeline.

Have u researched set theory

Scott Reynolds's picture

Have u researched set theory and Hierarchy ? its an interesting thing, and can become woefully inefficient without careful thought on the math to pull it. There is some great stuff in Pro MySQL book:http://www.apress.com/book/view/159059505X . Check your schools library and/or safari/ProQuest subscription.

If you would like to talk about the math ping me IRC: SupermanScott

Thanks a lot! I'll

Xano's picture

Thanks a lot! I'll gratefully take the offer :D

So the trick I like the most

Scott Reynolds's picture

So the trick I like the most and have talked about with a few people is called Nested Sets. I like this post even though I think its for SQL server.

http://www.codeproject.com/KB/database/nestedsets.aspx

The Pro MySQL book does a really cool job of describing it. I forget the book its based on though, or where the concept originated from. It basically adds two queries to each insert that update left and right values for the parent nodes in the tree. But with this you can arbitrary, without recursion, select all the nodes of a parent in one query. Its very slick for heavily queried data. It is something that if you are building a hierarchical set where you are selecting more then just one child node, this needs to be a part of the system. (unlike the menu system, where all we care about is the items direct child, not grandchildren).

Showing support

regmanabq's picture

Xano, i'm way behind on seeing this, but wanted to throw my support behind you and say you rock man. This has been needed for a long, long time!! I use the vocab index and still can't believe such functionality isn't in core, but hey, with smart young industrious men like you around that's not an issue.

I wish I could help more. I'll certainly help out in testing once I get my latest monster site whipped into shape.

Again, great job, can't wait to see it and hope things are coming along nicely!

Hierarchy and Faceted search

janusman's picture

I just found out about this effort; it sounds like it would be valuable to efforts like Apache Solr Search integration, Faceted Search module, Organic Groups (with subgroups?) and others.

What's the status? Do you need any help?? At least speaking as an Apache Solr contributor, I'd like to know more =)

Hierarchy API has been

Xano's picture

Hierarchy API has been renamed to Drupal Object Model. Index works without any other module and handles its own relationships.