The Drupal 8 committers have slightly changed the policy regarding automated test failures as we move toward release of 8.0.0. The traditional policy is that any proposed core patch must pass all automated tests at commit time. We are amending that slightly such that patches which solve Criticals, but fail tests in a non-essential component, are allowed to proceed. For now, the only non-essential component is the Migrate framework. As announced previously, the Migrate framework is not release blocking for 8.0.0 so it should not hold back these Critical patches.
When we do have a patch which is Critical but cannot easily be adjusted to pass Migrate's unit tests, we will
- File a postponed Major followup issue tagged 'Failing Migrate test'. Describe the test that was disabled and the challenge in making it pass. The issue should link to this post so that folks understand why the test was disabled. These issues become "Migrate-critical" (see [meta] Ensure that there are no critical Migrate issues before releasing the migration path).
- In your patch for the Critical issue, change the name of the failing test method from testFooBar() to failingTestFooBar() and add a @todo to its docblock referencing your follow up issue.
- If we ever get Add "expected fail" functionality to simpletest (help wanted!), we will use that to skip tests instead of changing the method name.
Before release, we will revisit all issues tagged with 'Failing Migrate test' and make a determination about Migrate's suitability for 8.0.0. If deemed unsuitable, Migrate (or a specific migration path) can be added back into 8.1.0 or later. See [meta] Ensure that there are no critical Migrate issues before releasing the migration path for more information.