Import Wizard - Mashup several import-related modules into one import profile

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
jasonn1234's picture

Synopsis:

No one content import module yet encompasses the vast array of use cases there are for content imports. So we should have a module that essentially allows for the mashing up those modules into multi-module profiles / macros which can all be managed through a common interface.

Problem:

Importing content into drupal is complex and - in the real world - usually requires a fair bit of custom coding to pull off.

There are already a ton of modules made for different use cases. Some very general, others more specific - but none of them provide a working, comprehensive solution (yet...). See http://groups.drupal.org/node/21338

There is also a great unmet need for import profiles and templates/presets for regular scheduled imports, although it's on the roadmap of a couple of the aforementioned modules.

If you have a site which requires alot of different imports from different sources, you are practically sentencing some poor schmuck web developer (ahem...) to spending all his time keeping track of the various imports that are needed and writing little custom scripts to make each one work, as well as maintaining multiple workflows which vary along with the different approaches and UIs of each import module.

Proposed Solution:

I've created an "Import Wizard" API which by itself is just a container made of the common steps for configuring any import process - from the data acquisition (file download, xml feed, upload, etc...) to the field mapping, and so on. It stays out of the "guts" of the import process altogether.

The crux of the implementation is that each step in a given import process can be handled by one of the many different contributed modules which satisfies in that particular step. But it can all be configured and then executed through one pluggable interface (similar to views 2).

    $steps = array(
      'name'  => t('Profile Name'),
      'schedule' => t('Schedule'),
      'file' => t('File'),
      'file_options' => t('File options'),
      'samples' => t('Data sample'),
      'type' => t('Content type'),
      'field_options' => t('Field options'),
      'field_defaults' => t('Defaults'),
      'map' => t('Field map'),
      'preview' => t('Preview'),
      'ingestion' => t('Ingestion status'),
    );

Each step, then, has multiple modules made available to the end user by means of a select menu. Changing the select menu in the "file" step, for example, from "Media Mover API" to "Feeds" changes the configuration form for that step and reveals whatever options for file/data acquisition the Feeds module makes available. In this case, it would allow the user to specify an XML feed as their data source (an option media mover doesn't have).

Conclusion:

The underlying philosophy behind the module is to provide a singular workflow tailored towards imports - all the while allowing for a great deal of variance among them by allowing an administrator to leverage any of the available import modules as needed.

Dev screenshot attached...

AttachmentSize
Picture 1 copy.png74.96 KB

Comments

This looks awesome.

anonymous07's picture

I have been pulling my hair out trying to find which module I should use to do batch CSV, TSV imports into Drupal 5 ... and well .... I am still trying to figure this out.

Any more info on your effort -- would it be compatible with Drupal 5?

No D5 port planned

jasonn1234's picture

Hi anonymous07... if anything I'll be working on a port to D7. Of course, by the time D7 is released there may be a "one true import solution" - Migrate looks promising - so this won't be needed.

Thanks Jason

anonymous07's picture

More and more it is looking like I have a "migration" in my near future, like it or not ... i.e., D5 -> D6 :-(

Thanks again.

database focused, but very general

greggles's picture

Hi Jason! I hope you're doing well.

The migrate module is focused on imports from databases, but is a very generalized tool: http://drupal.org/project/migrate

I'm betting that it will become the "one import tool to rule them all" over time with plugins that make it easy to import from a variety of different systems.

Contributed Module Ideas

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds:

Hot content this week