I wrote this module for the University of Ottawa (where I work) to help them in their migration to Drupal and I was wondering if people think it would be a good fit for a contributed module.
First off, I am aware of Migrate and Feeds - they're both really cool, but neither of them really addressed our use case. Feeds seemed far more interested in doing content aggregation and Migrate focuses on tasks where there is an easy 1 to 1 mapping of the data, such as importing a table of data or using an XML document as a source feed.
Both of them also require a developer to get involved to set up the import and we were looking at having a lot of different imports (because there are a lot of different content types coming in from our existing website).
My solution to this was to create a system where we could create an inventory of content to be imported and specify import paths for each one (or each section of the site, or just a global one). The import path takes the source URL, scrapes it with cURL, then provides a QueryPath object that can be used to map expressions to destination objects and fields (we provided a basic UI for this as well, so that you can set up a new mapping on the fly). Once the set of destination objects was created, we map each group of destination objects to a destination handler which is responsible for creating the object in Drupal itself.
While there's a number of features I'd still like to add to this to make it really cool, that's the barebones outline of what it does. It contains a lot of hardcoded decisions at the moment that we made for the University, so I would have to focus on making it very generic and configurable before I could contribute it, but I was curious what people thought of having this available as a contrib module.
Comments
Migrate
This sounds really similar to what Migrate already does, and the differences seem like things you could extend Migrate to do rather than reinventing some wheels. Migrate is incredibly modular, with nearly every part of it swappable for an alternate implementation, so it doesn't need to do exactly what you want to be a good place to start.
Heres a good overview on how Migrate can handle importing a static HTML site:
http://fourkitchens.com/blog/2012/05/04/migrating-old-html-files-drupal
Sounds like the only thing you're doing differently is loading the files with curl rather than locally and creating the file list with patterns. You should be able to do both in Migrate.
The advantage of using Migrate as a base is you don't have to figure out everything Migrate already does, like field mappings, rollbacks, and updates. Also, any improvements you make to Migrate can be used by the thousands of sites already using Migrate.
Hi, I think it would be great
Hi,
I think it would be great to be able to look at the source code of your module. I'm going to migrate a complex site to Drupal and Moodle and I'll probably develop some custom modules too.
Migrate is a good project, but seems not appropriate for what I need. On the other hand, creating a custom module to import that will help me to learn some Drupal internals.