Summary of the Data Architecture Design Sprint

Six Drupal developers met for three days in Feburary 2008 to work on and re-design Drupal's core data architecture. We began with an overly broad list of topics and goals and spoke in very general terms about "Drupal's overall mission" and similarly vague concepts. By the end of the design sprint, we devised a tangible vision for Drupal's future and an understandable method for getting there.

We believe that the overall goal for the Drupal community should be to convert Drupal from a monolithic Content Management System into a web application development platform in which a powerful CMS is one built-in implementation. We talked a lot about what that means and requires, including things like consistent and documented APIs for all parts of core, switching to an API-driven instead of form-driven architecture, and a data rendering architecture for producing output in many formats besides HTML. None of these are new ideas within the community, of course, but discussing them in the context of our stated goal (making Drupal into a web app development platform) provided a useful organizing structure and new insights.

One particular epiphany we had is the realization of how Drupal's existing architecture makes it an excellent tool for developing web applications in a web-services world. Drupal receives a request from an external source (currently, from a web browser or XML-RPC client), converts it to a query against a data store (currently, an SQL database), and returns the result. Before it returns the result, however, it exposes the data via Drupal's hook mechanism to an enormous variety of contributed modules that can add additional value to the data. The fact that so many disparate people can write modules to enhance content value through Drupal's hook interface is Drupal's key feature.

It became clear that, in a web services world, where Drupal needs to be able to both offer and consume web services, Drupal also needs to be able to provide its key feature---value-added data via contrib modules---to web services data. Comments, votes, maps, social bookmarks, etc. etc. etc., should be just as appliable to a content object retrieved from Amazon as a node retrieved from SQL.

We eventually decided, as we mostly knew we would, that moving CCK's field concept in Drupal core is important step forward. Keeping our two days of discussion about "the future of Drupal" in mind, we worked to define how to put fields in core in a way that is consistent with our proposed future and makes it easier to get there. We devised two "Proposed Content Models" for supporting what we called "multi-source" data with core fields though both need plenty more design work and exploration.

A lot of work still needs to be done and, as always with Drupal, most of it is currently slated to be performed by volunteers. It is our strong desire to at least have node-based fields in D7 core and, hopefully, have multi-source data support in D7 core as well. We will have to see what is achievable.