Interested in helping test Drupal 6 to Drupal 8 migrations? Start here for instructions on how to get set up, and where to start testing. Find us on IRC in #drupal-migrate.
We meet weekly on Thursdays at 1:00 UTC (Wed 6pm PDT, Wed 9pm EDT, Thurs 9am benjy-time). Post a comment on the event page if you'd like to participate (and not just watch) and we'll be sure to invite you when the hangout begins.
D6 > D8
All issues are in the core queue.
Pick any active issue, read the instructions, assign to yourself, and hack away! If you can't complete it, post a partial patch and unassign.
- Finish the remaining tests (intermediate skill)
D7 > D8
All issues are in the core queue.
WHAT: Import API in Drupal 8 core, based on the Migrate project. Also in core: Drupal 6 and 7 import using this API based on the Drupal-to-Drupal data migration project. Read and comment on Mike Ryan's thoughts at http://goo.gl/WnTOB0.
WHERE: If you'd like to be involved now, find us in IRC on #drupal-migrate. If you have any doubts about what to do, find us in IRC or post in the issue! Work takes place here: https://drupal.org/sandbox/chx/2105305. The UI is at https://drupal.org/sandbox/mikeryan/2271813 .
Migrate API documentation can be found at https://drupal.org/node/2127611
WHY: So that Drupal 6 sites can move directly to Drupal 8. (And because the current upgrade path is unmaintainable.)
HOW: We're currently doing targeted testing and patching of the D6->D8 migration path. Want to get involved - introduce yourself in #drupal-migrate on IRC!
Now broadcasting weekly meetings on Google+
Migration in core, aka IMP, returned to working in the sandbox last week. This is a slightly unusual development so some explanation is in order. The fundamental API and several migrations have been committed to core, establishing that this initiative actually has legs and will happen, but now that that's in place, it makes sense to return day-to-day work to the sandbox because:
- Core committers are (rightly) focused on beta blockers / criticals, so these patches sit around for a while before getting committed
- When the patches sit, they block other migration work
- The API is still young and we could use space and time to flesh it out more before submitting to core where it becomes more "locked down"
- Arguably, the patches will be easier to review as a whole, because then we can test the D6 -> D8 migration itself, and not just piecemeal parts of it.
We will continue to use the core queue for:
- Major adjustments to the migrate API already committed to core
- Bug fixes to the core API
If you have questions, post them to this discussion or find us in IRC in #drupal-contribute.Read more
Since we decided to remove support for the Drupal 7 to Drupal 8 upgrade path, update hooks are no longer being added consistently and it no longer makes sense to add update hooks that will never be run. This document should be used to document things that would have required update hooks and may now need data migrations.
See https://drupal.org/node/2168011 for more information.
Variables to state or config
(list issues here with a brief note)
Added or removed database fields and tables
Not all schema changes require data migration.Read more
The December holiday season is traditionally a little slower than the rest of the year, but it didn’t stand in the way of steady progress for IMP!
Major milestone achievement/s:
The first complete config entity migration, #2150407 Migrate D6 roles and permissions, was committed to core (Thanks, alexpott!). This patch introduced the entity destination in a very minimal state, the first two process-specific plugins, and a unit-tested source plug in as well.
Steady progress:Read more
You’ll need clones of Drupal 6, 7, and 8. They don’t have to be installed, just cloned.
D6: The install file so you can look at hook_schema implementation
D7: The *.install files can help you locate more information about what changed between 6 and 8 in the update functions.
D8: The entity class
We're collecting questions from today's sprint in hopes of getting documentation together for future work.
Where do I put my tests?
Add tests to the exisiting test files.
For MigrateExecutable.php, add to:
For Sql.php, add to
What does the getMock function do? What's the high-level idea of the function and how do I know what to pass to it?Read more
Getting set up to contribute to IMP
- Visit the IMP sandbox Version control tab and follow the instructions to clone the project. The sandbox contains both Drupal 8 core and the migrate module so it takes a bit to download.
- Work for this project is located in
core/modules/migrate. If you have access to commit directly to IMP, commit directly to this 8.x branch. Otherwise, create and submit patches to the IMP issue queue. Directions are on the Version control tab under the heading Patches. Note: chx will handle consolidating work into patches for core as necessary.
- It is helpful to download and install Drupal 6 in a separate directory so you can test enabling modules, configuring them, creating content and seeing how that content and configurations are stored in the D6 database.
Backing up a bit to a high-level view of the architecture, right now we've got the whole thing in a single Migrate module. I'd like to see some separation between the APIs that will service all kinds of migrations, and the support specific to Drupal-to-Drupal upgrades, as well as identifying what (if anything) requires a module. The buckets of code I picture:
- Migrate API: Most of what we have now.
- Upgrade API: Any classes specific to Drupal-to-Drupal migration.
- Upgrade module: D6/D7 migration configurations, plus UI.
- migrate_plus module (or even call it "migrate", if there's no module in core?): General migration UI, D7 migrate features not going into core such as different source plugins, etc.
- migrate_d2d module: Support for D5 and D8 sources, UI for remapping content types (i.e., support for custom Drupal-to-Drupal migrations as opposed to upgrade-all-the-things), etc.
The IMP team is working hard to complete a core migration path from Drupal 6 to Drupal 8 by the end of January 2014. Join us on November 19 between 9:00am - 5:00pm Pacific for our second sprint.
The IMP initiaitve lead, chx, will be guiding developers from all over via IRC in freenode's #drupal-contribute channel.
Portland-based OpenSourcery has dedicated a team of three outstanding developers to support a face-to-face sprint in downtown Portland, Oregon. If you're in town, consider joining us! https://groups.drupal.org/node/368358
The sprint goal is complete unit test coverage for
It's a great chance to get involved with Drupal 8 and get up to speed with PHPUnit in core.
- Among many things, chx is working on the FakeInsert, FakeUpdate, and FakeMerge classes so SQL writes in the id_map can be unit tested.
- Others are creating a slew of migrations and integration tests for variables-to-config. Detailed instructions are available, so you can just grab an issue without much previous knowledge. Reviews are happening swiftly (less than 24 hours) and chx and others are highly available in IRC to answer question.
- Late next week, we'll set up the sprint-day issues. If you'd like to lend a hand with that, let me know in the sprint planning issue.
filter_update_7003 is reading settings from a variable for each format it knows about. How do we handle this? Obviously we can cook this into prepareRow of the source class; but then how will contrib get on board? Reminder: we plan to use the entity load hook to let contrib extend the (module,delta) -> map mapping. It would make quite some sense to use the same hook as the event contribs will need to fire on but I am not sure what exactly they would change! Probably we need to provide some sort of extensibility for prepareRow apparently just not sure how. Another type of plugin??
Where we are:
- The process syntax has been revamped completely, the new get has a test and there's documentation
- There's an entity handler but completely untested (as in I never ran it)
- There's a map process plugin but it's also quite untested
- bdone has written a few variables-to-config migrations (thanks!) others haven't, we are badly behind on them
- Reroll core patch
- In import, handle the case where $destination->import succeeds but there are no ids -- as in config
- Write more tests: migration and map process plugins, sql idmap. sql idmap tests should go into core patch #1
- Write the filter formats migration
- Create core patch #2 with the map process plugin, the filter formats source, the entity destination the filter formats migration and make sure all of this is tested
- Get people to write the variables-to-config migrations somehow.
- Entity destinations need the stub functionality. Not urgent.
Join Acquia and DOOR3 for an informative webinar on why migrating to Drupal CMS provides the flexibility and ROI your organization desires. With a strong practice around both .NET and open source (and Drupal), we understand all of the pros, cons and realities when it comes to CMS migrations. You'll hear an informative discussion on why this is the last move you’ll ever have to make.Read more
Currently the process values are a list of plugins and their configurations. This is causing a lot of headaches because of ordering. And also, noone thinks in terms in plugins. The Drupal 7 version of addFieldMapping was also taking destination as the first argument. Reading the current documentation at https://drupal.org/node/2127611 first might be helpful.Read more
As drupal.org will be down the document is copied to https://docs.google.com/document/d/1ubNnzQf38jp_yPKixg-x1DeiVoRvow9dLIqY... and the code is pushed to http://github.com/chx/imp
- Almost all of the D6 sources are done
- marvil07 is leading an effort to add requirements checking, in the beginning they are just modules existing and being the right schema version
- There's a new Source class, it's not pluggable, it wraps the source plugin.Same code as in SourceBase just separated out.
- The message display is in a separate class, it even has an interface, mandatory to pass to MigrateExecutable. Default implementation is just drupal_set_message().
- I am working on the first migration, id_map needs to in a semi decent shape.
What needs to be done:
- Review the current state with core in mind. Add missing doxygens.
- Once the first variable-to-config migration is done, we need to get another 50 of those
- Entity destinations. The base should be blatantly simple.
- url_alias destination.
So far we have established the structure of the new migrate module, there is a migration configuration entity which holds the configuration of the various plugins: source, process, destination and id map. As Views does, migrate also will have a
MigrateExecutable class, responsible for "turning the gears" -- calling the proper plugins. There is a standalone, non-pluggable
Row class which mostly just holds the source and destination data and does very little logic.
We need to version the sources for sure. I can't remember a case when a core minor release would break our queries but such things definitely happened in contrib (look at CCK it keeps the table names in a function because of this).Read more
When I look at SourceBase, it looks like a standalone class to me which wraps an iterator from the real source base. Should we split it off instead of having these funny named methods like performRewind, getCurrentRow? Our database classes already return an iterator, it's trivial to wrap reading a file in an iterator ( https://github.com/weppos/fileiterator/blob/master/FileIterator.class.php ) so I think this would be beneficial. Also we could skip computeCount and just use count() and make the sources implement Countable.Read more
I need a developer with skills below to work on a 9-month project in Arlington, VA.
US Citizens/Green Cards only. No H1B
Must undergo a background investigation followed by a drug test, we pay for these
Job Title : Drupal Develper
Location: Arlington, VA
Duration: 9 months
• Experience with Drupal 6/7 - knowledge of all major contributed modules including Views, Panels, CCK, etc
• Custom module development experience
• Mastery of CSS / HTML / PHP
• PHP programming experience: 5 years
• Familiarity with Git
• Comfortable with LAMP Stack
Yesterday jessehs was writing the filter formats plugin and we had a discussion live and then on IRC with marvil07 whether we should migrate the filter format table and then run a migration for each format to bring the filters over or just add the filter list to the formats in prepareRow. We are doing the latter.
Noone suggests bringing over entities and then fields separately either.Read more
As far as I am concerned, the UI will contain:
- the database credential fields straight out of the installer
A Drupal version select boxit seems we can divine this from the system table.
- A file path textfield. (MIght need two for private files in D7)
- A button saying Migrate configuration
- A button saying Migrate content