I have been conducting some research into how other systems are organized and manage their configuration and content deployment. There are plenty of systems that don't manage this well or whose problems are similar to ours, but I've found a couple places that have some interesting approaches that I am going to start adding here. Please note that my experience with these systems is limited to several hours of research each, and that is just barely enough to be scratching the surface, especially since many of these systems are so different architecturally. If anyone with more experience finds inaccuracies or has more to add then I encourage you to do so. Additionally, if anyone has other interesting systems to check out, please let me know or add them yourself!
The one commonality seems to be that these systems have a very hard line drawn between what is content and what is configuration. This obviously reduces their challenges. Many of the systems I have looked at also implement UUIDs for content references and other uses. Neither Plone nor Alfresco use an RDBMS (Plone uses the ZODB object database, and Alfresco stores content on the file system) so many of the concerns we have about performance don't apply (or are at least different.)
For now this is just somewhat of a note dump, will try and clean it up later but I went to start pushing out what I was learning as I learned it.
Plone 4 User Manual and a reasonable overview
Collections are Views. Portlets are blocks.
Usage of UUIDs for content reference
- Three schemas aka objects - persistent data, form data, config data. All extend the interface class.
Some high level feature overviews
By default, Alfresco has chosen to store meta-data in a database and content in a file system. Using a database immediately brings in the benefits of databases that have been developed over many years such as transaction support, scaling & administration capabilities. Content is stored in the file system to allow for very large content, random access, streaming and options for different storage devices.
A lot of good info here about how their DM deployment stuff works under the hood
Specifics about resolution of remote content
UI side of the deployment
Content replication screencast
- Can push content from one source to multiple 'transfer targets'
- Remote content is read only
- Can specify batches of data to go to each target, can be different per target
- All defined via UI
- Pretty similar to Deploy, can select folders of content to send, scheduling
- Replication jobs run in the background
- Replicates deletions too
- Entire transfer is a single transaction and if any part fails the whole thing rolls back
- Very strong line between content and config
- Implements three key foundation services on which everything is built (Node (config), Content, Search), all share the same transaction, security and configuration characteristics.
- They are using UUIDs plus some other identifying information, but internally they also appear to be using int IDs for primary and foreign keys
Zend Config is a set of class provided by the Zend Framework for managing configuration information. It is pretty interesting and provides a lot of concepts we can probably steal and use. These classes are available in both ZF1 and ZF2, and the 2.0 remains pretty much unchanged to my eyes from the 1.0 version, except for the addition of Writer classes (more discussion below.)
At its most basic level, Zend Config provides an OO interface to associative arrays. The Zend_Config object takes an array in its constructor, and stores it internally. This class is Countable and Iterable. It then provides a set of functions to interact with it. For instance, you can use magic get() and set() functions but it also implements its own internal get() and set() functions to allow more options (for instance, the internal get() function allows you to specify a default just like variable_get() does.) If you pass in a nested array, then the constructor will recursively turn it into a tree of Zend Config objects, with one object at every branch. These are chainable so you can easily do things like
$foo = $config->views->advanced->disable_cache;
$config->views->advanced->disable_cache = TRUE;
By default the config objects are read-only, but this can be changed when the object is constructed, and it can be changed as granularly as you want (you can make Views' advanced settings be read-only while leaving the rest of them read-write.)
In addition to this basic operation, there are classes that extend the Zend_Config base class in order to be able to hook it up to files of different types. For instance there is Zend_Config_Ini, Zend_Config_JSON, etc. These are constructed by passing a filename instead of an array, but otherwise they behave the same way, simply adding the formatting and reading information necessary for the specific formats. Initially there were no classes to later write this info back out to disk, but there is now a Zend_Config_Writer class with extended classes of the same types as the readers.
One really interesting aspect of this class is that you can provide granular 'overrides' to specific values. For instance, take this sample INI file (stolen from the Zend wiki at http://framework.zend.com/wiki/display/ZFUSER/Zend_Config+Example):
; Staging environment
host = staging.example.com
db.type = pdo_mysql
db.host = localhost
db.username = someuser
db.password = somepass
db.name = somedb;
Production environment inherits values from staging environment and
; overrides values that are specific to the production environment
[production : staging]
host = www.example.com
db.host = db.example.com
db.username = anotheruser
db.password = anotherpass
You can now do the following:
$config = new Zend_Config_Ini('/path/to/config.ini', 'staging');
echo $config->host; // prints "staging.www.example.com"
echo $config->db->host; // prints "localhost"
echo $config->db->name; // prints "somedb"
$config = new Zend_Config_Ini('/path/to/config.ini', 'production');
echo $config->host; // prints "www.example.com"
echo $config->db->host; // prints "db.example.com"
echo $config->db->name; // prints "somedb"
These overrides can also nest multiple levels, extending off each other as far as you like.
A final cool feature is that Zend Config provides merge functionality. You can take two Zend_Config objects and merge them. Any non-existent items from the array being merged-in will be added, same named items will overwrite. This can provide some really useful functionality in terms of overriding functionality between installations while still inheriting most settings.
Some Examples and Tutorials: