Distribution export module

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
shaisachs's picture

I'm in the midst of putting together my own installation profile, and only now realized the profile wizard is gone. What a shame!

I'm now working on a module which will allow you to export a lot of settings from an existing D6 site into code suitable for consumption by the install profile API. That is, it will generate the code needed to recreate vocabs, content types, views, etc. The idea is that you install D6, install the modules you want in the profile, configure them, create vocabularies, etc., and then go to /distrib_export, which will create the necessary PHP code to feed to installation profile API in your .profile module's _final function.

I'm attaching what I have so far, which is not much at all; right now it only covers vocabularies, and is rather sparse on the UI. But I wanted to toss it out and see whether this is the right direction, if it's the kind of thing people are looking for, and/or if it would conflict with other efforts to migrate the profile wizard.

As an aside, in the future I think there should be some kind of "export API" which allows modules to consciously export their export-able data for purposes like these. Such an export API could also be used to produce a kind of lightweight Drupal backup, analogous to mysqldump's structure-only option. I know the idea of an export API is problematic - what counts as export-able? - but I think that for some modules it would be obvious, and for others, there could be settings to allow the site admin to choose what is and isn't export-able.

Any thoughts?

Thanks!

AttachmentSize
distrib_export.info_.txt158 bytes
distrib_export.module.txt1.03 KB

Comments

This is a great start for

jlmeredith's picture

This is a great start for this tool! I will be playing with what you have built thus far and will report back any thoughts. Please continue to post your updates!

Jamie

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

Go for it

Boris Mann's picture

I'm a maintainer for install_profile_api. No one stepped up to port the wizard AND it makes sense, as a module, to just have it be a separate project. So ... if you want to take this on, GO FOR IT.

Also, yes, all the folks looking at install profiles / patterns / etc. etc. all decided to work on ONE THING together for D7 -- which would be a core import / export hook. Then each module can implement that hook, rather than us having to support all modules in some other piece of code.

Please tell us what support you need from install profile API module to make your work easier...

I disagree about the second part

dmitrig01's picture

but i'd rather have this discussion in private

the_base install profile

George2's picture

i started to create an install profile that was for my needs with my essential modules etc. then i started to think about importing my essential cck / views, and so wrote a little function that just pulls in .inc files from specific dirs and installs them. my vision was then to write an export module to write the dumps from a live site to separate files that could then be put into such an install profile format.

this would also allow admins to store their own content types / views in the sub dirs, and - with a bit of modification to the profile - to then choose what content types, views, blocks etc to install during the install process via checkboxes (if an advanced mode was selected, say). it would also allow admins to modify and quickly swap include files about for a new site!

the_base is currently not using the install profile api. however, in the next update it will be as it's a no-brainer to use it as an .inc for the profile - though personally i don't like the idea of having it as a required module.

i'd be up for the challenge of an exporter to go hand-in-hand with the_base.

No but good luck

dmitrig01's picture

This is a great idea but is simply impractical unless you want to write integration with every contrib module that exists. I tried my hand at thus, and trust me I got a lot farther along than you are, but it didn't work because I found that I was running into that problem.

The other problem I wanted to address was: What if I want to export nodes X and Z but not Y? How am I going to provide an interface/method to do that? How am I going to do the same thing for every other feature every other contrib module implements?

Another question that came up - what if created some item that had some sort of ID that isn't the same every time, say menu items, and then I create some sort of other object which references the ID. How do I solve that problem both in code and in the interface?

Anyway, good luck!

Yeah - there was a BOF at

shaisachs's picture

Yeah - there was a BOF at Drupalcon Boston about this subject. Short answer, it's a massive pain in the neck. You can work around it in a piecemeal fashion (getting a NID dynamically using a compound pseudo-key on content type/title/etc), but there's no really good solution. I guess if we had a really solid slug infrastructure, similar to Wordpress's, and slug uniqueness and persistence was guaranteed, then we could use those instead of DB ids, which really aren't suited for persistence and uniqueness when you're dealing with a production/staging/development architecture.

But as far as "export nodes X and Z but not Y" - couldn't you just write a module to allow node authors to check off "export this node" in the admin or edit form, or use Flag alternatively, and then just export those flagged nodes during export? That seems like a reasonable approach.

Anyway, if you have some code that's better than the admittedly junky stuff I've been posting, I'd love to see it!

yeah thanks for the

George2's picture

yeah thanks for the positives dmitri!

in essence, the problem you outlined above - simply impractical unless you want to write integration with every contrib module - is exactly what the installation profile api tackles, but for import. you'd have to be a fool to dl this new crazy module, then expect to be able to extract it and import it...but you would be able to write an export function for such an 'extraction api' just as you can write an import function for this installation api.

of course i foresee problems, yes i'm toying with solutions. if you wanted to export nodes x/y but not z....views integration?..ie. bulk ops has a lot of similarity. and for creating different ids, yep, difficult, but not impossible i think.

-

dmitrig01's picture

in essence, the problem you outlined above - simply impractical unless you want to write integration with every contrib module - is exactly what the installation profile api tackles, but for import. you'd have to be a fool to dl this new crazy module, then expect to be able to extract it and import it...but you would be able to write an export function for such an 'extraction api' just as you can write an import function for this installation api.

There are two problems with this - first of all, install profile api is never going to be able to export all stuff from everyone, and second, what about variable_sets and such? How do you know which variables are set? More importantly, how do you know that they are different from the default without hook_variable_defaults?

This gets to the bigger issue - it'd be even more insanely hard to make a profile exporter where the people don't know how to code and don't know the internals of drupal. When letting them export, how do you provide a human readable name of the variable that makes sense (ucfirst(str_replace('_', ' ', $variable_name)) just won't cut it - "User register" isn't exactly a readable name). Then, if you want to show what they are actually set to, you'd need to know what every value for every variable means - user_register is a yes/no, but the cache type is 0, 1, or 2.

of course i foresee problems, yes i'm toying with solutions. if you wanted to export nodes x/y but not z....views integration?..ie. bulk ops has a lot of similarity. and for creating different ids, yep, difficult, but not impossible i think.

I don't think views integration would work, because then you'd need views to be able to select anything (think menu items, etc).

Different ID's - i'm really curious as to how you would do this. The problem (and solution) with/of an ID is it is tied to a type of object (e.g. menu item #1) and within that object it's really unique. I guess if you were really sensitive to various contexts then it would work, but then you'd need so much code it'd be insanely hard (e.g. the mlid parameter of menu items is the ID of another menu item, and the way you get the ID of a menu item after saving is getting $item['mlid'] but you can't do it before saving).

Check out 'features'

alex_b's picture

The features work that yhahn explains in this post: http://groups.drupal.org/node/20119 has a lot of overlap with this proposal.

http://www.twitter.com/lx_barth

shaisachs's picture

Thanks everyone for your comments. I'm sure that this module is not a really good, industrial-strength solution moving forward. That will require an import/export API in Drupal core and some diligence on the part of many module contributors. Still, it's better than clicking copy-and-paste-and-click again and again, so I'm pressing forward, at least to the point that it suits my own needs in building distribution profiles. To that end, I've added support for content profiles, and cleaned up the output a little bit; see the attached module. This module doesn't faithfully export your CCK groups, because I don't need that, nor does it export changes to the Page or Story type. If you want to post changes or improvements, or if you want to post a harness application that pulls together distrib_export and install_profile_api to quickly and programmatically recreate a Drupal distribution, go for it!

Also, below is a very simple script I've written to allow me to test my install profile easily on a MAMP distribution. The idea is to automate the process of deleting settings.php, copy default.settings.php over to settings.php, drop and recreate the database, etc. It's quite a simple script, but I figure it can't hurt to share it:

chmod u+w /Applications/MAMP/htdocs/drupal/sites/default/
chmod u+w /Applications/MAMP/htdocs/drupal/sites/default/settings.php
rm /Applications/MAMP/htdocs/drupal/sites/default/settings.php
cp /Applications/MAMP/htdocs/drupal/sites/default/default.settings.php /Applications/MAMP/htdocs/drupal/sites/default/settings.php
echo "drop database drupal;" > reset_drupal.sql
echo "create database drupal;" >> reset_drupal.sql
/Applications/MAMP/Library/bin/mysql -u USERNAME -pPASSWORD < reset_drupal.sql
rm reset_drupal.sql

Finally I would add that there's some great work being done by Development Seed that is related to all of this, which I haven't had a chance to check out in much depth but which, based on their Drupalcon 2008 presentation, sounded really exciting. See the Context (http://drupal.org/project/context) and Spaces (http://drupal.org/project/spaces) modules for more information.

Distributions

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: