Looking for suggestions on how to migrate files from remote server

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
webchuck's picture

I am in the process of migrating a large collection of media assets, namely images, documents, and video thumbnails (video is already stored locally) from the Kaltura platform to be stored locally in Drupal. I'm an experienced Drupal developer, but new to the Migrate module. I could just write a custom module that does this, but I feel like the Migrate module may be the better way to go. Unfortunately, I'm just not sure how this would work...

Here are the steps I know I need to take.

  • Pull all assets from the remote servers and store them locally (I'm assuming this should be done first before we migrate)
  • Run a migrate import that creates file entities for all the new files
  • Update existing field collections to link to the file entity (these field collections currently store the Kaltura ID)

Any help on what order and how these should be handled with the Migrate module would be awesome. Specifically, how are the file entities in step 2 linked to the field collection entity in step 3?

Comments

Go with migrate!

Ryan Weal's picture

What you'll probably want to do is code a custom migration module. The hard part is getting used to OOP if you're new to that, and then just making sure to register the classes the right way and the construction of those classes has all the things you need. After that it is just field mappings.

I have done a few presentations on this, my last slides are probably worth looking at:

http://www.verbosity.ca/getting-content-drupal-migrate-montr%C3%A9al

For field_collection you might need to do some custom code to get things into the shape you need on the D7 site... but you have generic destination table classes that can be used to manage the importing process (plus rollbacks, etc).

Comparison of import/export modules

Frank Ralf's picture

You might have a look at this comparison table http://groups.drupal.org/node/21338 but I second Ryan, use Migrate, especially together with Drush.

hth
Frank