The install profile API became the most important helper for building Drupal profiles. It provides a library of create functions that supports the most important Drupal modules and provisions a place for contributing more create functions.
Problems:
- In the Drupal 5 version, install profile API contains a wizard that generates install profile stubs and defines export functions for some of the API's install functionality, in Drupal 6 there is no successor for install profile wizard yet.
- When building install profiles, in many cases, portions of an existing prototype site need to be exported in an install profile API compatible way.
- At the same time, deploy module defines functionality for exporting and importing site configuration. This creates the potential for unnecessary overlap at least on the import side of things.
Proposed improvement:
I think there is a need for a structured way of creating site elements that not just addresses installing but also generating install functions. Moreover, such a structured way of creating site elements should be shared between modules like install profile API or deploy.
I propose therefore to move install profile API's install_ functions to a new module called port that not just holds install functions, but their export function twins. Every install function should have a corresponding export function that generates its arguments. A simple hook should be used to declare which export function matches which install function.
This functionality should live in a new module, because it provides general import/export functionality that goes beyond an install profile API. For the same reason, the terminology would change from "installing" to "importing" as far as port module is concerned.
Install profile API would continue to contain functions that are aimed at helping building Drupal site structure from scratch - not with elements exported from prototype sites.
Having export/import pairs available in the application makes it trivial to create an install profile wizard that can generate install profiles from existing sites that contain the entire site configuration defined by available export/import pairs.
I've coded a Drupal 5 proof of concept that is available in my sandbox http://cvs.drupal.org/viewvc.py/drupal/contributions/sandbox/alex_b/port/ . It contains export/import support for
- module status
- menus
- image cache
- node profile
- site variables
- user roles and permissions
- content types
These elements can be exported (all or partial export) to executable PHP functions for install profiles or ported with copy/paste from text area to text area.
Concrete steps:
1) Break out install functions into port module, rename them from _install to _import, write import/export declarations, write export functions where trivial, release refactored version of install profile API, release port module (all D6).
2) Write install profile wizard on top of port module's API
Looking forward to your feedback.
Comments
Loving it! but why no API's?
Great idea! there are already some import modules (nodes, ubercart has his product import through xml).
Why aren't the drupal API's used when inserting the data into the newly created database? For example to create a menu item in menu.inc there is a sql statement for the insert while in my opinion the API menu_save_item($item) should be used.
We should create a full list of all database entities with there export and import functions.
Marnix.
Using API's
We should work on that. But this is not the point of the proposal - port is really just about declaring matching export / import function pairs. It's entirely up to the implementation whether it uses API functions or direct SQL statements.
http://www.twitter.com/lx_barth
http://www.twitter.com/lxbarth
Blocks, menus, content types, fields, and views
This is something we've been thinking about as well --
Importing/exporting blocks is tricky because of the various ways a block can be generated --
For example, let's say we have a block derived from a menu that contains three links: one of these is a standard page; one is a node created using cck fields; one is generated by a view.
To export this block, we would need to get the menu, which means we also need to get:
for the standard page: the data for the page -- easy.
for the node extended using cck fields: the node type, and all added fields, which in turn sets up some dependencies on additional modules, such as the actual cck fields, and possibly helper modules like autonodetitle or token -- more complex.
for the link created by the view: the view definition (pretty straightforward), but then any node types/field definitions used by the view, where it gets similar to #2.
And this doesn't say anything about users (ie, controlling what user in the new site has access to newly imported pages), roles (if a view has role-based access control) --
It seems like an install profile quickly becomes context specific -- we have been thinking about using helper modules to generate the install, and have different helper modules generate the context into which various elements are added.
Which is a long way of saying I'm completely stoked that you started this thread -- it's great timing :)
Cheers,
Bill
FunnyMonkey
Tools for Teachers
FunnyMonkey
How does this problem fit in?
Is there anything in the port module that makes solving the block related problems you're talking about more difficult/easier? Could there something in a port module like API that could make it easier to solve these problems? If so, what would it be?
http://www.twitter.com/lx_barth
http://www.twitter.com/lxbarth
More on this later...
But some quick thoughts now --
First off, the idea of import/export pairs seems like a Very Good Thing -- views and cck already does this, and it's incredibly useful.
The reason I mentioned blocks and menus is that they frequently rely on elements from many disparate pieces of a drupal site --
As we've been looking at it, menus and blocks are at the core of a site, and as such often represent a context-specific solution -- ie, this specific navigation/IA supports this specific user story. So, importing/exporting menus can require importing/exporting some or all of the underlying functionality that goes into generating the menu -- in some cases, this can include a view, content type definitions, individual nodes (and possibly attached content), user roles, and possibly even individual users --
So, the question becomes how much of this context is actually required for a successful import/export? Where is the line?
FWIW, I also see this as part of the dev --> staging --> deploy issue, and we are looking at install profiles both because we want cleaner install profiles for work we are doing, but we also would love to get some of this functionality to help with developing and deploying sites.
Like I said, more to come later, and also this is something we are thinking about, and have some ideas about, but definitely not something we feel we have solved -- so in some cases, I'll be thinking out loud, and I'm very interested in what other folks have to say about this.
Cheers,
Bill
FunnyMonkey
Tools for Teachers
FunnyMonkey
Bill your idea of helper
Bill your idea of helper modules is something I am looking at, too.
Maybe a module could interview the person installing, ask questions about they want, and then install combos based on answers to those questions, for instance?
I am working on something a bit like this over the next couple of months.
Sam Rose
Social Synergy
Open Source Ecology
P2P Foundation
Sam Rose
Hollymead Capital Partners
P2P Foundation
Social Media Classroom
Tracking this...
Hi Alex -- I have this on my "to do" list to respond to in detail. The short answer is that I'd like to see a really clear line around install profile API and port. And especially around using "core" Drupal calls as well (and contrib, for that matter ... encourage contrib to put those files in their own package, not having to duplicate ... the goal should be to get rid of the .inc files that WE have to maintain).
Some of the _install functions really ARE install functions -- not import. So where is the line?
Does install profile API go away completely? I've already said that I'm kicking profile wizard out -- it has to be rebuilt on D6 and depend on install / port / whatever.
I'd like to take steps that moves us closer to helping core or whatever, rather than building a system outside it. So, build on the outside for quick movement, but look closely at the evolution of install profiles / packaging / etc.
That was longer than I meant, but also not quite long enough to cover all the issues. More as time permits...
Port should be an incubator
These are the functions that would stay in install profile API. Further, port wouldn't usually define import/export functions, just declare which export function matches which import function. Of course, to move forward, we will have to define some of these import/export functions in the port .inc files - such definitions could very quickly become a patch to their respective modules though.
Exactly... when building port, I was really trying to come up with an incubator for standardized import/export functionality in Drupal. Much in the spirit of install profile API. I am looking forward to your thoughts and suggestions in regards to this angle.
Of course, as long as port isn't core, what remains in port .inc files is the declaration of import/export function pairs...
http://www.twitter.com/lx_barth
http://www.twitter.com/lxbarth
Port on existing sites
The problem at the moment with the install profiles is that you need to start on a new site, you cannot "add" an install profile to your existing site. Does port help in such a situation?
One example is the Kendra Hub KDI proposal which provides install profiles but also "update profiles" that one can install on his site without starting from scratch. In the end, whether you're creating a new site or adding a set of functionality to an existing site should not matter.
I too am looking for this
I too am looking for this "update profile" functionality.
see my post Can I use an install profile to update an existing site?
M.M.
"The problem at the moment
"The problem at the moment with the install profiles is that you need to start on a new site, you cannot "add" an install profile to your existing site. Does port help in such a situation?"
It does...
http://www.twitter.com/lx_barth
http://www.twitter.com/lxbarth
Next steps?
So, it sounds like the plan from DCDC was to submit hook_import and hook_export (as part of system.module? not sure...) for core inclusion. If that is the plan, then it makes sense to start a new project for D6 that is essentially a back port of that system.
I wasn't there, so not sure who is taking the lead on this?
Check out Packaging and Deployment
One result of DCDC was the packaging and deployment group http://groups.drupal.org/packaging-deployment
Check out Bhuga's notes from the BoF on import/export, deployment and packaging: http://groups.drupal.org/node/20063
yhahn and jmiccolis are working on some packaging stuff called 'features': http://groups.drupal.org/node/20119
Aside from a really cool way of exporting configuration from your website, 'features' is building heavily on putting configuration into code i. e. like views export supports this.
This is a game changer to my proposal here, because while it provisions support for exporting things into code, it follows more of a 'scripting' approach. I. e. 'port' won't allow you to revert to the version in code like e. g. views does that.
I think this views style managing configuration in code is most desirable for deployment.
Check out young's screencast here: http://groups.drupal.org/node/20119 - this has also big implications for install profiles wizard - If 'features' become widely adopted, they can take the place of install profile wizard.
http://www.twitter.com/lx_barth
http://www.twitter.com/lxbarth