A brief summary of the idea:
Data within entities that do not need to be processed individually within a view (or better: are not required to be addressed individually in any case) should be stored in a single database table, rather than being spread over N tables like they would if stored by the Field API "the usual way", in order to reduce the number of (sub)queries or subsequent queries and hereby reduce the load on the site's database.
In a recent client's project - a big and heavily visited recipe site - we stumbled upon massive performance issues partially caused by the huge number of fields attached to a specific node type. As part of the result set I had the idea to reduce the number of "real" FAPI fields by storing them in a serialized manner in the database, thus reducing the number of queries made to the site's database.
In detail: in the recent scenario we had a field collection containing 5 different fields placed inside a field collection containing an additional field. Each of the field collections was set to allow unlimited numbers of entries. Additionally, two of these were actually Term- and Entity reference fields causing a bunch of subsequent queries on their own...
This data structure led to a huge number of complex queries containing joins and generating even more subsequent queries to load even a single node. But - you may guess it already - these nodes weren't only displayed "one by one", in fact they were aggregated quite often using views, which caused the situation to get even worse... Regarding the fact that the main part of the fields contained were of no use without the containing node entity itself brought up the idea to serialize these fields and to reduce the number of queries needed to load a node significantly. This idea was implemented as a custom tailored solution and it turned out to work very well. So I set out to create a more generic and reusable version of this technique.
Create a new custom field type able to hold the serialized data, provide a hook system for submodules to hook into in order to provide new field types, provide a field settings widget to create "virtual" fields and groups, an entity edit widget providing a simple, intuitive and user-friendly interface to enter the data "the usual way" and finally a field formatter to properly display the field where it should get displayed.
I started developing this about 2 weeks ago and have gone pretty far up to now...
Key facts of the actual development state:
- The custom field type: created using Drupal's Field API utilizing a blob field with up to 4 GByte of data.
- The hook ecosystem: fundamentally outlined and implemented.
- The field settings widget: ready to use, already calling and implementing sub-modules.
- The entity edit form widget: ready to use, offering form validation, error markings and messages and a centralized autocomplete feature for sub-modules to easily hook into.
- The field formatter: yet to be done, actually just outlined and roughly implemented.
Existing sub-modules / field types:
- (Plain) Text fields: existing and working.
- Text areas: roughly outlined and implemented.
- Taxonomy Term reference: existing and working with a select and an autocomplete widget.
- Node reference: roughly outlined and implemented.
- User reference: roughly outlined and implemented.
- Options (checkboxes, radio sets and select fields): planned.
- Numbers: planned.
- File- and image uploads: planned.
Like to check it out?
If you would like to check it out, here is the link to the sandbox. Or the git clone ready to go:
git clone --branch 7.x-1.x-dev http://git.drupal.org/sandbox/clamine/1997240.git virtual_field_collection
I would love to hear what you think!