Export/import content with Features

antoniodemarco's picture

Hi all,

I’m planning to contribute a module which will offer nodes import export capabilities for features, let me know what you think about it, if it makes sense to have something like this etc...

Here its description:

Features Content

The module enables Features to export/import nodes objects (along with their files, menu items and CCK fields) in a feature package. For each feature it is possible to decide which node to export (based on its NID, type, etc...) and if to keep its NID or to recreate it from scratch to the next feature import/update. For the moment authorship export is not supported: all the imported nodes will inherit a common user UID (configurable) if their own is not present in the system.

Drush Integration

Drush integration is provided too to allow a quick content export/import simply running:

  • drush content export: export content to a file
  • drush content import: import content from a file
  • drush content clean: utility to clean up content (based on a .info configuration file)

Options include the --file flag to specify the name and location of the dump PHP file.

Use Case 1: Features with default content

The module allow a feature to ship with default content ready to be imported. Possible usages could be:

  • Dummy example content
  • Support pages
  • Credits and others

Use Case 2: Push changes to production

The module could be useful also in case the maintainer needs to push some configuration from development to production. Here the workflow:

  1. run drush content export
  2. push new DB dump
  3. run drush content clean
  4. run drush content import

Implementation

As proof of concept only the Drush integration is working at the moment, although a complete Features integration is on its way. The feature’s .info file module options look like this:

features[features_content][nid][] = 253
features[features_content][nid][] = 254
features[features_content][nid][] = 255
features[features_content][nid][] = 256

features[features_content][type][] = book
features[features_content][type][] = article

The node is the driving force of the module, the export of related items depends on feature wide configuration options, probably looking like:

features[features_content][user settings][export] = 0
features[features_content][user settings][default user] = 1

Comments

Sounds cool! Will you work

mig5's picture

Sounds cool! Will you work out a way to throw in taxonomy support :) I believe there's some issue in Features module with lack of exportables for some things like taxonomy?

For the moment node's

antoniodemarco's picture

For the moment node's taxonomy terms are imported if their vocabularies are already present in the website but no taxonomy creation is supported. I think this falls out of the scope of the module itself even if a taxonomy integration would be useful and complementary to it. It is anyway in part of the Features roadmap: http://www.developmentseed.org/blog/2009/may/29/making-and-using-feature...

Thanks for commenting.

import / export framework

miro_dietiker's picture

I'm very interested about import / export tools.
BTW: isn't package & deployment the wrong group?
My use cases often differ a little. It's much more about importing or sync from previous CMS DB or a filebased dump. These cases often have a need of custom import code. However one could add hooks to universalize the approach.

I would love to support a fully featured toolset with coding to standardize these processes instead of code imports from scratch again and again...

Hi, According the first use

antoniodemarco's picture

Hi,

According the first use case I pointed out (Use Case 1: Features with default content) this is the right group according the second one: you are right. I think the module could be interesting for other groups too, I posted only here (for the moment) to keep the focus on its integration with Features since I'm already using it to push configuration keeping the content untouched (even if only on the same site etc... so quite a simple scenario).

Deploy

heyrocker's picture

Have you looked at Deploy?

Project page: http://drupal.org/project/deploy
Screencast: http://www.youtube.com/watch?v=7PjwT0HWHxw

While Deploy does not have Drush support right now, it does a great job of pushing nodes around and managing their dependencies. Maybe you could contribute a patch which adds Drush support?

I've checked Deploy module

antoniodemarco's picture

I've checked Deploy module and it's indeed very interesting. How do you see it in relation to the two use cases I've posted above? For example, since Deploy already manages content export and dependencies, it would be maybe interesting to build a Features integration for it. This allows to package content into a feature and not only transferring it using RPC/XML.

Content poses some unique

heyrocker's picture

Content poses some unique problems that I don't believe Features is prepared to handle. The biggest problem, for instance, is that when you push a node from one server to another, the nid is likely going to change (or alternately, you are likely going to run into an ID conflict.) This problem is actually at the core of everything Deploy does - it manages keys and foreign keys by creating UUIDs that transfer between servers, and then maps them between production and dev on deployment. For this to happen you really need a lot of communication between the server, and a simple package and push concept doesn't really work. The problems posed by content vs the problems posed by configuration data are very different.

One thing I have had some requests for is to make the transfer layer for Deploy pluggable. In this case you could move your info between servers however you want, not just xmlrpc. Sam Boyer has this dream of allowing this mechanism to happen using git, for instance. If this happened, then lots of more interesting integrations could happen including possibly Features. However, again, this is a somewhat complicated issue and it's not something I'm approaching anytime soon.

While you're on the thread, heyrocker...

bhuga-gdo's picture

I looked into exactly this kind of context export/import a while back and never got to it; we were looking at features and would have needed to export several nodes types (like feed api nodes, for example). I think the use case is different than deploy's, in which you're talking about possibly sending this to third parties, or deploying to multiple sites with similar configuration.

However, I would like to point out one thing: doing this without deploy_uuid is almost ridiculous, because you'd have to rewrite that module to do anything useful. There's a stand-alone uuid module but it's not really very feature-complete right now and deploy_uuid is 'there'. It also has no dependencies, although it's a sub-module of the deploy project.

Heyrocker, please consider breaking uuid out of the deploy framework into its own project? I think it would be a very useful dependency for a variety of projects, but the deploy framework itself is not everyone's use case and is a pretty sizable thing. There is some deploy-specific code in there but it's not bothering me any, and it's too awesome to let deploy hoard it :)

I originally wanted to break

heyrocker's picture

I originally wanted to break UUID out into its own project, possibly integrating with the existing UUID project, and I just didn't have time to get it done for the client who sponsored all this work. I'd like to revisit it sometime, I'm not very happy with how all that stuff is bolted together at all, I just haven't gotten around to it. I don't want to break it out into its own project until I've sorted all that out. In the meantime, as you note, it has no dependencies and there's nothing stopping anyone from integrating it as is.

There has also been a lot of discussion about integrating UUIDs into Drupal core. While it now seems unlikely for this to happen in Drupal 7, I'd honestly rather see people (myself included) work towards that goal than bolting on half-finished solutions.

Features+CTools

alex_b's picture

Features does have a drush integration. The commands would be sth like:

drush features export [feature]
drush features update [feature]

There is no 'import' as the export is a feature module that can be just enabled on the target site (= imported). I wonder whether CTools/export could help you handle node exports. It may have problems with scaling, but it could be worth a try. If you wind up going with CT/export, features integration for it is here: http://drupal.org/node/532256

Also make sure you check out deploy.

You're in a field that's moving wildly and it's worth having a good look at some of the existing solutions out there.

http://www.twitter.com/lx_barth

Hi Alex, the Drush

antoniodemarco's picture

Hi Alex,

the Drush integration provided with the module is not meant as a replacement of the Features one but it's just there because I actually started to develop the module as a Drush extension :).

About CTools: thanks for the hint, I've checked it out and it is indeed a very useful tool but it looks like it is only able to export table records without carrying along any Drupal logic: is that true? Am I missing something?

Content Export (the module I'm working on) exports full node objects being able, at import phase, to reconstruct the entire data structure of a node (CCK fields, taxonomy terms, etc...). Still there is something that is related to a node and it's not included in its object, for example a menu item. On this extend I'm interested in the Deploy module since it already handle all this logic (node relations, taxonomy terms, etc..); maybe a Features integration it's possible there. If not, then I'll probably need to have an extensible architecture on top of which it will be possible plug content export extensions that will be able to export what a node object is not carrying along itself.

The reason why I see Features as a good place to implement a filesystem-based content dump is that every feature is (or at least is meant to be) a complete environment of settings and configurations in which a node object could be safely placed and carried around, without running into the risk of missing requirements or dependences.

"without carrying along any

alex_b's picture

"without carrying along any Drupal logic:"

If you mean executable code - not directly. Not a good idea either. But I see why you're looking for it: e. g. CCK requires some scripting logic to create fields etc. Features module has therefore its own meta format for exporting CCK information. Can be read in includes/features.content.inc . When you're creating not content types but nodes itself, you can find yourself in the same situation quickly.

"Still there is something that is related to a node and it's not included in its object, for example a menu item. "

Features does have dependencies, so you could add a menu (not sure about menu item) to the dependency specification of your feature.

In regards to deploy module, I'll have to let heyrocker speak for that, as I am not that familiar with it. However, I am sure that researching the pro/cons of the two modules would be very worthwhile in your situation. Would love to read your findings, too ;-)

http://www.twitter.com/lx_barth

"without carrying along any

alex_b's picture

"without carrying along any Drupal logic:"

If you mean executable code - not directly. Not a good idea either. But I see why you're looking for it: e. g. CCK requires some scripting logic to create fields etc. Features module has therefore its own meta format for exporting CCK information. Can be read in includes/features.content.inc . When you're creating not content types but nodes itself, you can find yourself in the same situation quickly.

"Still there is something that is related to a node and it's not included in its object, for example a menu item. "

Features does have dependencies, so you could add a menu (not sure about menu item) to the dependency specification of your feature.

In regards to deploy module, I'll have to let heyrocker speak for that, as I am not that familiar with it. However, I am sure that researching the pro/cons of the two modules would be very worthwhile in your situation. Would love to read your findings, too ;-)

http://www.twitter.com/lx_barth

how is it going? this seems

dasjo's picture

how is it going? this seems quite interesting, tell me when we can test something.
josef

Hi josef, at the moment I'm

antoniodemarco's picture

Hi josef,

at the moment I'm developing the module in my local environment: the API structure and export/import functionalities are kind of complete, what is missing is the actual integration with the feature module. Soon I'll have an use case to internally test the module: bundling in features nodes that are used to store configuration. Some modules already use this approach, for example FeedAPI or Complete.

I think this will be the focus of its first release (a simplified/proof-of-concept version): being able to bundle in features single nodes not covering, for the moment, dependencies or other complexity.

FeedAPI doesn't support

alex_b's picture

FeedAPI doesn't support features - won't happen any time soon either, as FeedAPI stores much of its configuration in variables. I use frequently http://drupal.org/project/strongarm for exporting variables.

Care to put up your code on github?

Subscribe

bdwelle's picture

Hi folks

We've just deployed the first phase of a pretty big (at least for us) Drupal project: http://admonsters.com/

We're using drush for migration / deployment and especially now that phase 1 is live I'm feeling the lack of drush content export/import, so I'd love to help move this along if I can. It would be very useful to be able to export/import a single node or a range of nodes based on some criteria (where field = x). I'm certainly very interested in testing when something is available.

thx

-- Bowen

Interested in this too

redben's picture

We have a use case of "space templates" where a feature, if enabled, creates some default content

Watching :)

Would be nice!

pimousse98's picture

Would be nice!

I want to help out with this

kristofvanroy's picture

I want to help out with this project as well. It's very interesting.

Table Wizard the Migrate

emptyvoid's picture

For data manipulation and transforms the following set of modules are very useful. Unfortunately the modules don't cover the scope of the elements you are proposing (media, menu items). If anything the current build has a very useful GUI that you may want to take usability pointers from.

Migrate
http://drupal.org/project/migrate

Table Wizard
http://drupal.org/project/tw

I would recommend building additional sub-modules that add media and menu items as resources for the Migrate module.

Robert Foley Jr
Solutions Architect
http://www.robertfoleyjr.com

Help you for your module

InfernalAura's picture

Hi antoniodemarco,

Me and my friends are very interessting about your module. We can help you to finish it.

Just ask me how can we help you.

Thanks

InfernalAura

uuid_features

mori's picture

http://drupal.org/project/uuid_features - did you check out this module? Unfortunately there´s no D7 version available yet.

There is now a D7 version.

mirzu's picture

There is now a D7 version.

Should this module exist?

CarlHinton's picture

Should this module even exist? Many in the Drupal community will agree that in many instances it actually breaks the basic concept of keeping data in the database and logic in the codebase. I agree that a view maybe described as logic (sometimes), but since when could content be described as logic? Content belongs in the database

Webforms...

arpieb's picture

I could definitely see a need where Webforms are concerned. These are not simply dynamic content, but instead provide functionality to the site and are fairly static, not unlike core contact forms...

Packaging & Deployment

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: