Hi all,
I’m planning to contribute a module which will offer nodes import export capabilities for features, let me know what you think about it, if it makes sense to have something like this etc...
Here its description:
Features Content
The module enables Features to export/import nodes objects (along with their files, menu items and CCK fields) in a feature package. For each feature it is possible to decide which node to export (based on its NID, type, etc...) and if to keep its NID or to recreate it from scratch to the next feature import/update. For the moment authorship export is not supported: all the imported nodes will inherit a common user UID (configurable) if their own is not present in the system.
Drush Integration
Drush integration is provided too to allow a quick content export/import simply running:
- drush content export: export content to a file
- drush content import: import content from a file
- drush content clean: utility to clean up content (based on a .info configuration file)
Options include the --file flag to specify the name and location of the dump PHP file.
Use Case 1: Features with default content
The module allow a feature to ship with default content ready to be imported. Possible usages could be:
- Dummy example content
- Support pages
- Credits and others
Use Case 2: Push changes to production
The module could be useful also in case the maintainer needs to push some configuration from development to production. Here the workflow:
- run drush content export
- push new DB dump
- run drush content clean
- run drush content import
Implementation
As proof of concept only the Drush integration is working at the moment, although a complete Features integration is on its way. The feature’s .info file module options look like this:
features[features_content][nid][] = 253
features[features_content][nid][] = 254
features[features_content][nid][] = 255
features[features_content][nid][] = 256
features[features_content][type][] = book
features[features_content][type][] = articleThe node is the driving force of the module, the export of related items depends on feature wide configuration options, probably looking like:
features[features_content][user settings][export] = 0
features[features_content][user settings][default user] = 1
Comments
Sounds cool! Will you work
Sounds cool! Will you work out a way to throw in taxonomy support :) I believe there's some issue in Features module with lack of exportables for some things like taxonomy?
For the moment node's
For the moment node's taxonomy terms are imported if their vocabularies are already present in the website but no taxonomy creation is supported. I think this falls out of the scope of the module itself even if a taxonomy integration would be useful and complementary to it. It is anyway in part of the Features roadmap: http://www.developmentseed.org/blog/2009/may/29/making-and-using-feature...
Thanks for commenting.
import / export framework
I'm very interested about import / export tools.
BTW: isn't package & deployment the wrong group?
My use cases often differ a little. It's much more about importing or sync from previous CMS DB or a filebased dump. These cases often have a need of custom import code. However one could add hooks to universalize the approach.
I would love to support a fully featured toolset with coding to standardize these processes instead of code imports from scratch again and again...
Hi, According the first use
Hi,
According the first use case I pointed out (Use Case 1: Features with default content) this is the right group according the second one: you are right. I think the module could be interesting for other groups too, I posted only here (for the moment) to keep the focus on its integration with Features since I'm already using it to push configuration keeping the content untouched (even if only on the same site etc... so quite a simple scenario).
Deploy
Have you looked at Deploy?
Project page: http://drupal.org/project/deploy
Screencast: http://www.youtube.com/watch?v=7PjwT0HWHxw
While Deploy does not have Drush support right now, it does a great job of pushing nodes around and managing their dependencies. Maybe you could contribute a patch which adds Drush support?
I've checked Deploy module
I've checked Deploy module and it's indeed very interesting. How do you see it in relation to the two use cases I've posted above? For example, since Deploy already manages content export and dependencies, it would be maybe interesting to build a Features integration for it. This allows to package content into a feature and not only transferring it using RPC/XML.
Content poses some unique
Content poses some unique problems that I don't believe Features is prepared to handle. The biggest problem, for instance, is that when you push a node from one server to another, the nid is likely going to change (or alternately, you are likely going to run into an ID conflict.) This problem is actually at the core of everything Deploy does - it manages keys and foreign keys by creating UUIDs that transfer between servers, and then maps them between production and dev on deployment. For this to happen you really need a lot of communication between the server, and a simple package and push concept doesn't really work. The problems posed by content vs the problems posed by configuration data are very different.
One thing I have had some requests for is to make the transfer layer for Deploy pluggable. In this case you could move your info between servers however you want, not just xmlrpc. Sam Boyer has this dream of allowing this mechanism to happen using git, for instance. If this happened, then lots of more interesting integrations could happen including possibly Features. However, again, this is a somewhat complicated issue and it's not something I'm approaching anytime soon.
While you're on the thread, heyrocker...
I looked into exactly this kind of context export/import a while back and never got to it; we were looking at features and would have needed to export several nodes types (like feed api nodes, for example). I think the use case is different than deploy's, in which you're talking about possibly sending this to third parties, or deploying to multiple sites with similar configuration.
However, I would like to point out one thing: doing this without deploy_uuid is almost ridiculous, because you'd have to rewrite that module to do anything useful. There's a stand-alone uuid module but it's not really very feature-complete right now and deploy_uuid is 'there'. It also has no dependencies, although it's a sub-module of the deploy project.
Heyrocker, please consider breaking uuid out of the deploy framework into its own project? I think it would be a very useful dependency for a variety of projects, but the deploy framework itself is not everyone's use case and is a pretty sizable thing. There is some deploy-specific code in there but it's not bothering me any, and it's too awesome to let deploy hoard it :)
I originally wanted to break
I originally wanted to break UUID out into its own project, possibly integrating with the existing UUID project, and I just didn't have time to get it done for the client who sponsored all this work. I'd like to revisit it sometime, I'm not very happy with how all that stuff is bolted together at all, I just haven't gotten around to it. I don't want to break it out into its own project until I've sorted all that out. In the meantime, as you note, it has no dependencies and there's nothing stopping anyone from integrating it as is.
There has also been a lot of discussion about integrating UUIDs into Drupal core. While it now seems unlikely for this to happen in Drupal 7, I'd honestly rather see people (myself included) work towards that goal than bolting on half-finished solutions.
Features+CTools
Features does have a drush integration. The commands would be sth like:
drush features export [feature]drush features update [feature]
There is no 'import' as the export is a feature module that can be just enabled on the target site (= imported). I wonder whether CTools/export could help you handle node exports. It may have problems with scaling, but it could be worth a try. If you wind up going with CT/export, features integration for it is here: http://drupal.org/node/532256
Also make sure you check out deploy.
You're in a field that's moving wildly and it's worth having a good look at some of the existing solutions out there.
http://www.twitter.com/lx_barth
http://www.twitter.com/lxbarth
Hi Alex, the Drush
Hi Alex,
the Drush integration provided with the module is not meant as a replacement of the Features one but it's just there because I actually started to develop the module as a Drush extension :).
About CTools: thanks for the hint, I've checked it out and it is indeed a very useful tool but it looks like it is only able to export table records without carrying along any Drupal logic: is that true? Am I missing something?
Content Export (the module I'm working on) exports full node objects being able, at import phase, to reconstruct the entire data structure of a node (CCK fields, taxonomy terms, etc...). Still there is something that is related to a node and it's not included in its object, for example a menu item. On this extend I'm interested in the Deploy module since it already handle all this logic (node relations, taxonomy terms, etc..); maybe a Features integration it's possible there. If not, then I'll probably need to have an extensible architecture on top of which it will be possible plug content export extensions that will be able to export what a node object is not carrying along itself.
The reason why I see Features as a good place to implement a filesystem-based content dump is that every feature is (or at least is meant to be) a complete environment of settings and configurations in which a node object could be safely placed and carried around, without running into the risk of missing requirements or dependences.
"without carrying along any
"without carrying along any Drupal logic:"
If you mean executable code - not directly. Not a good idea either. But I see why you're looking for it: e. g. CCK requires some scripting logic to create fields etc. Features module has therefore its own meta format for exporting CCK information. Can be read in includes/features.content.inc . When you're creating not content types but nodes itself, you can find yourself in the same situation quickly.
"Still there is something that is related to a node and it's not included in its object, for example a menu item. "
Features does have dependencies, so you could add a menu (not sure about menu item) to the dependency specification of your feature.
In regards to deploy module, I'll have to let heyrocker speak for that, as I am not that familiar with it. However, I am sure that researching the pro/cons of the two modules would be very worthwhile in your situation. Would love to read your findings, too ;-)
http://www.twitter.com/lx_barth
http://www.twitter.com/lxbarth
"without carrying along any
"without carrying along any Drupal logic:"
If you mean executable code - not directly. Not a good idea either. But I see why you're looking for it: e. g. CCK requires some scripting logic to create fields etc. Features module has therefore its own meta format for exporting CCK information. Can be read in includes/features.content.inc . When you're creating not content types but nodes itself, you can find yourself in the same situation quickly.
"Still there is something that is related to a node and it's not included in its object, for example a menu item. "
Features does have dependencies, so you could add a menu (not sure about menu item) to the dependency specification of your feature.
In regards to deploy module, I'll have to let heyrocker speak for that, as I am not that familiar with it. However, I am sure that researching the pro/cons of the two modules would be very worthwhile in your situation. Would love to read your findings, too ;-)
http://www.twitter.com/lx_barth
http://www.twitter.com/lxbarth
how is it going? this seems
how is it going? this seems quite interesting, tell me when we can test something.
josef
Hi josef, at the moment I'm
Hi josef,
at the moment I'm developing the module in my local environment: the API structure and export/import functionalities are kind of complete, what is missing is the actual integration with the feature module. Soon I'll have an use case to internally test the module: bundling in features nodes that are used to store configuration. Some modules already use this approach, for example FeedAPI or Complete.
I think this will be the focus of its first release (a simplified/proof-of-concept version): being able to bundle in features single nodes not covering, for the moment, dependencies or other complexity.
FeedAPI doesn't support
FeedAPI doesn't support features - won't happen any time soon either, as FeedAPI stores much of its configuration in variables. I use frequently http://drupal.org/project/strongarm for exporting variables.
Care to put up your code on github?
http://www.twitter.com/lxbarth
Subscribe
Hi folks
We've just deployed the first phase of a pretty big (at least for us) Drupal project: http://admonsters.com/
We're using drush for migration / deployment and especially now that phase 1 is live I'm feeling the lack of drush content export/import, so I'd love to help move this along if I can. It would be very useful to be able to export/import a single node or a range of nodes based on some criteria (where field = x). I'm certainly very interested in testing when something is available.
thx
-- Bowen
Interested in this too
We have a use case of "space templates" where a feature, if enabled, creates some default content
Watching :)
Would be nice!
Would be nice!
I want to help out with this
I want to help out with this project as well. It's very interesting.
Table Wizard the Migrate
For data manipulation and transforms the following set of modules are very useful. Unfortunately the modules don't cover the scope of the elements you are proposing (media, menu items). If anything the current build has a very useful GUI that you may want to take usability pointers from.
Migrate
http://drupal.org/project/migrate
Table Wizard
http://drupal.org/project/tw
I would recommend building additional sub-modules that add media and menu items as resources for the Migrate module.
Robert Foley Jr
Solutions Architect
http://www.robertfoleyjr.com
Help you for your module
Hi antoniodemarco,
Me and my friends are very interessting about your module. We can help you to finish it.
Just ask me how can we help you.
Thanks
InfernalAura
uuid_features
http://drupal.org/project/uuid_features - did you check out this module? Unfortunately there´s no D7 version available yet.
There is now a D7 version.
There is now a D7 version.
Should this module exist?
Should this module even exist? Many in the Drupal community will agree that in many instances it actually breaks the basic concept of keeping data in the database and logic in the codebase. I agree that a view maybe described as logic (sometimes), but since when could content be described as logic? Content belongs in the database
Webforms...
I could definitely see a need where Webforms are concerned. These are not simply dynamic content, but instead provide functionality to the site and are fairly static, not unlike core contact forms...