Data API

OK, so, it took a lot longer than I expected or planned, but here is the final summary report from the design sprint. I've provided it in both OpenDoc and PDF format. Hopefully there aren't too many spelling errors. :-)

During the Data Architecture Design Sprint (or DADS, as I will henceforth call it), we discussed two general models for reaching our vision of "data anywhere, value-add anything". Those models became dubbed "Model 1" and "Model 2", because that's the order in which we happened to write them down.

Barry has already done a good job of explaining Model 1 in an earlier post. Here, I am going to lay out the structure of Model 2. I suspect that the final system, whever it looks like, will draw heavily from both models.

My primary battle plan should be enhance Drupal cross database compatibility. I have involved in this topic for around a year, and I would like to keep it on going. To accomplish this target, I would like to complete the following tasks before D7 code freeze:

• Improve Database API for better abstraction

  Something should be done before any other targets, e.g. resolve reserved words conflict, split drivers into individual files for better management, indeed but simple abstraction for different database syntax, add INSERT/UPDATE/DELETE abstraction and enhance drupal_write_record() abstraction, etc. With a better common base for multiple database development, we will need much less time to study the differences between databases than that of before.

• Add PHP PDO supporting

  Shipping D7 with PHP PDO should be a spotlight topic, since it is now official package for PHP5.2.x, and legacy drivers will soon be removed in PHP6.x. We may need some cleanup in our core query syntax, in order to let PDO get works.

• Introduce more database backend

  Up to this point, I would like to introduce some other database backend for D7, e.g. Oracle, SQLite, IBM DB2 and also MSSQL. If we are able to support SQLite in D7, which also means we are going to standardize our core queries into SQL92 standard, the problem for maintain multiple database backend should no longer a critical problem; if we are able to support Oracle, the most complicated database I guess, this should no longer a problem for us to implement drivers for other databases.

I have summarize most of my personal battle targets in here. Most logic are proved as functioning, and they are all get set for the open of D7 public development. On the other hand, I would like to explore if this may integrate with other interest musings with Data API, so we will able get all stuff better in D7 ;-)

Re-posting this as its own thing, per Barry. Admins, feel free to edit the crap out of this. ;)

Jan 17 - Jan 20
We get our collective stuff together internally... figure out a rough agenda, logistics about where/when we're meeting, compile resources, etc.

Jan 21
This group becomes the "Data API" group and is opened to the public.

I have been pondering the question of a data API for a while, as have a lot of people. Much of the recent discussion has focused on an Active Record approach to a data API. Now, Active Record is a very powerful architectural pattern. It maps nicely from storage to interface, it can be fairly self-documenting, and it is conceptually simple and approachable.

It is also, I believe, insufficient.

As mentioned on the other paper A Data API for Drupal. Here is a paper that shows how this could be accomplished with some OOP bits implementing the Active Records Pattern. While keeping the interface procedural.

We all know that Dupal will not go full OOP. But some OOP code could help us do things not possible with procedural code. Such as lazy loading of nested objects and 'on demand' database fields processing.

There's growing interest in the Drupal community in the prospect of renewing our core data handling APIs. Doing so will increase consistency and efficiency and ease barriers. It will also be a key step in enabling transactional web services.

In Drupal 6 we took some impressive first steps. What should we tackle for Drupal 7?

The attached paper, written by Nedjo Rogers and Henrique Recidive and sponsored by CivicSpace, aims to carry this discussion forward and map out both some conceptual space and concrete development tasks.