>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #2735059. --> Reported by: [gábor hojtsy](https://www.drupal.org/user/4166) >>> <h3>Background</h3> <p>Starting with Drupal 8, major version upgrades are done with a migration path, rather than an upgrade path. Both 6 -&gt; 8 and 7 -&gt; 8 migrations are included in core. While migrations apply to moving from 3rd party systems and other use cases, this meta issue deals with older Drupal to newer Drupal migrations only. Per <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/ideas/issues/2858592" title="Status: Closed (fixed)">#2858592: *DRAFT* Proposed product goals for Drupal 8.4/8.5(+) core</a></span>, this is the top product goal for Drupal 8.4.</p> <h3>Goals</h3> <ol> <li>Provide a stable Migration API in core. (<a href="https://www.drupal.org/node/2859225">Core <em>Migrate</em> module marked stable</a>.)</li> <li>Resolve all "<a href="https://www.drupal.org/project/issues/search/drupal?project_issue_followers=&amp;status%5B0%5D=Open&amp;version%5B0%5D=8.x&amp;issue_tags_op=%3D&amp;issue_tags=Migrate%20critical&amp;text=&amp;assigned=&amp;submitted=&amp;&amp;&amp;&amp;order=created&amp;sort=asc">Migrate critical</a>" issues. These include a working Drupal 6 to Drupal 8 migration, a working Drupal 7 to Drupal 8 migration, and a shippable Drupal Migration UI. (Core <em>Migrate Drupal</em> and <em>Migrate Drupal UI</em> modules marked stable.)</li> </ol> <h3>Proposal roadmap</h3> <p><span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/drupal/issues/2456259" title="Status: Closed (fixed)">#2456259: [META] Drupal 7 to Drupal 8 Migration path</a></span> to fix and introduce migrations from earlier Drupal versions.</p> <p>Triaged migration issues are listed at <a href="https://docs.google.com/spreadsheets/d/16SwQsp7nF1UKrZQCNdPYML_wi53ZjM1egwuLoPDwzUY/edit#gid=1071206918">https://docs.google.com/spreadsheets/d/16SwQsp7nF1UKrZQCNdPYML_wi53ZjM1egwuLoPDwzUY/edit#gid=1071206918</a> (<code>migrate triage?</code> in IRC). This breaks things down in more detail.</p> <h3>Remaining work</h3> <p>These are roughly the priorities of remaining work:</p> <ol> <li><span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-1"><a href="https://www.drupal.org/project/drupal/issues/2748609" title="Status: Active">#2748609: [meta] Preserving auto-increment IDs on migration is fragile</a></span> - An inherent consequence of Drupal's reliance on serial IDs is that attempting to preserve the "old" IDs in migration means that any attempt to create manual content in the destination system before migration is complete can cause collisions and data loss. It is critical to figure out how best to address this problem. A meeting is being scheduled for the week of March 20-24 to discuss this among the migration team and key core personnel.</li> <li><span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/drupal/issues/2683435" title="Status: Closed (fixed)">#2683435: CCK does not exist in Drupal 7, rename Migrate's cckfield plugins to field plugins</a></span> - Because of the volume of code touched, and the desire to have other patches immediately start using the new APIs, this blocks a bunch of work due to fear of reroll hell.</li> <li><span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/drupal/issues/2447727" title="Status: Closed (fixed)">#2447727: Add base class for migrating reference fields</a></span> - Blocked by the above patch, this blocks several reference field migrations.</li> <li><span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/drupal/issues/2820490" title="Status: Closed (fixed)">#2820490: FormatDate process plugin</a></span>, <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/drupal/issues/2566779" title="Status: Closed (fixed)">#2566779: Migration D6 &gt; D8 of CCK date fields</a></span> - Date fields, along with the afore-mentioned reference fields, are the most important gap in Drupal-to-Drupal migration functionality.</li> <li><span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/drupal/issues/2569805" title="Status: Closed (fixed)">#2569805: For Drupal migration, identify the source module</a></span> - Replaces the hardcoded list of migration source and destination modules with an automated method of determing them - getting this in means further backend issues no longer have to touch frontend code.</li> <li><a href="https://www.drupal.org/project/issues/search/drupal?project_issue_followers=&amp;status%5B0%5D=Open&amp;version%5B0%5D=8.x&amp;issue_tags_op=%3D&amp;issue_tags=Migrate%20UI&amp;text=&amp;assigned=&amp;submitted=&amp;&amp;&amp;&amp;order=field_issue_priority&amp;sort=desc">UI issues</a></li> <li><a href="https://www.drupal.org/project/issues/search/drupal?project_issue_followers=&amp;status%5B0%5D=Open&amp;categories%5B0%5D=2&amp;version%5B0%5D=8.x&amp;issue_tags_op=%3D&amp;issue_tags=migrate-d6-d8%2Cmigrate-d7-d8&amp;text=&amp;assigned=&amp;submitted=&amp;&amp;&amp;order=field_issue_priority&amp;sort=desc">Completing D6/D7 upgrade funcionality</a></li> <li><a href="https://www.drupal.org/project/issues/search/drupal?project_issue_followers=&amp;status%5B0%5D=Open&amp;priorities%5B0%5D=400&amp;priorities%5B1%5D=300&amp;categories%5B0%5D=1&amp;version%5B0%5D=8.x&amp;component%5B0%5D=migration%20system&amp;issue_tags_op=%3D&amp;text=&amp;assigned=&amp;submitted=&amp;issue_tags=&amp;order=field_issue_priority&amp;sort=desc">Fixing critical and major bugs</a></li> <li><span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/drupal/issues/2561243" title="Status: Closed (fixed)">#2561243: [meta] Migration system documentation</a></span></li> </ol> <h3 id="meet">Communicating with the team <a href="https://www.drupal.org#meet">#</a></h3> <p> Members of the team can be found in #drupal-migrate in <a href="http://drupal.org/irc">Freenode IRC</a> and #migration in <a href="https://drupal.slack.com/">Drupal Slack</a>.</p> <p>Weekly migrate meetings are held in google hangouts every week alternating between 2PM and 9PM <a href="https://www.timeanddate.com/time/zones/gmt">UTC</a> on Thursdays, focusing on triaging core 'migration system' issues. See the <a href="https://calendar.google.com/calendar/embed?showTitle=0&amp;showPrint=0&amp;showhthttps://www.drupal.org/about/strategic-initiativestps://www.drupal.org/about/strategic-initiativesTabs=0&amp;showCalendars=0&amp;mode=MONTH&amp;height=300&amp;wkst=1&amp;bgcolor=%23FFFFFF&amp;src=drupalcorecalendar@association.drupal.org&amp;color=%231B887A&amp;ctz=Etc/GMT&amp;pli=1">Drupal Core Calendar</a> for details. Please ping heddn in IRC to be added to the invite if interested in attending.</p> <p> <a href="https://www.drupal.org/project/issues/search/drupal?project_issue_followers=&amp;status%5B%5D=Open&amp;component%5B%5D=migration+system&amp;issue_tags_op=%3D">The migration system component is used to file all issues</a>.</p> <h3>Not in scope</h3> <p> Issues not related to older Drupal migrations to newer Drupal versions.</p> <h3>Related work</h3> <h4>Existing core features</h4> <p> Core contains four modules related to migration:</p> <ul> <li><strong>Migrate</strong>: The underlying API for the migration system.</li> <li><strong>Migrate Drupal</strong>: Underlying API for Drupal-to-Drupal migrations, currently including migration paths from Drupal 6 and Drupal 7 to Drupal 8</li> <li><strong>Migrate Drupal UI</strong>: Provides a very simple user interface for running migrations from older versions of Drupal (similar to update.php).</li> <li><strong>Migrate Drupal Multilingual</strong>: Needs to be enabled for multilingual migrations.</li> </ul> <p> It should also be noted that migration support for each individual core module is contained within that module, not within the migrate modules, although the multilingual support may still be considered experimental.</p> <h4>Open core issues</h4> <p> @todo (This would be things NOT in Migrate queues, but affected/related. E.g. the URL case sensitivity upgrade path issue.)</p> <ul> <li><span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-13"><a href="https://www.drupal.org/project/drupal/issues/2906470" title="Status: Needs work">#2906470: Orphan comments and entries in comment_entity_statistics after comment field instance has been deleted</a></span>. Required because otherwise migrations generate duplicate content to comment_entity_statistics table when usong D8 Standard and there's no way to clean the duplicates.</li> </ul> <h4>Contributed projects</h4> <p> The following are contributed projects in the migration system ecosystem, none slated for core inclusion at this time:</p> <ul> <li><a href="https://www.drupal.org/project/migrate_plus">Migrate Plus</a> provides additional functionality to the core migration system, such as example migrations.</li> <li><a href="https://www.drupal.org/project/migrate_upgrade">Migrate Upgrade</a>: Provides the drush equivalent to migrate_drupal_ui (runs full older-to-newer Drupal migrations only). See <a href="https://github.com/drush-ops/drush/issues/2140">https://github.com/drush-ops/drush/issues/2140</a>.</li> <li><a href="https://www.drupal.org/project/migrate_tools">Migrate Tools</a> General-purpose Drush commands for running and managing migrations.</li> <li><a href="https://www.drupal.org/project/migrate_manifest">Migrate Manifest</a>: provides a Drush command for running migrations using a manifest format (listing specific migrations).</li> <li><a href="https://www.drupal.org/project/migrate_run">Migrate Run</a>: A fork of migrate-tools that uses core plugin discovery and does not depend on migrate_plus.</li> </ul> <h3>Team and resources</h3> <ul> <li><a href="https://www.drupal.org/u/phenaproxima">Adam Globus-Hoenich (phenaproxima)</a></li> <li><a href="https://www.drupal.org/u/mikelutz">Michael Lutz (mikelutz)</a></li> <li><a href="https://www.drupal.org/u/heddn">Lucas Hedding (heddn)</a></li> <li><a href="https://www.drupal.org/u/quietone">Vicki Spagnolo (quietone)</a></li> <li><a href="https://www.drupal.org/u/g%C3%A1bor-hojtsy">G&aacute;bor Hojtsy</a></li> <li><a href="https://www.drupal.org/u/masipila">Markus Sipil&auml; (masipila)</a>, documentation maintainer</li> </ul> <p> @todo: Define team members' roles here (e.g. initiative coordinator, etc.; see <a href="https://www.drupal.org/contribute/core/maintainers#initiative;">https://www.drupal.org/contribute/core/maintainers#initiative;</a> compare <a href="https://www.drupal.org/node/2721129">https://www.drupal.org/node/2721129</a>). List additional team members; there are many active Migrate contributors in addition to the maintainers that might be part of the team.</p> <h3>Signoffs</h3> <h4>Signoffs needed</h4> <p> For any significant change: There needs to be a majority of Migration subsystem maintainers who agree to a given patch.</p> <p> For most significant changes, including API changes and new migrations (?): Migration patches also need sign-off from framework managers.</p> <p> For user-facing changes to Drupal Migrate UI, additionally the usability and accessibility topic maintainers, and product managers should be involved, since this is a critical site builder UI.</p> <p> For any changes that are disruptive, that increase the technical debt/followups needed, or that make existing issues more significant: Release managers should sign off.</p> <h4>Signoffs given</h4> <p> The core committers already signed off on replacing the old update.php-based upgrade system with a migration system at DrupalCon Prague in 2013, and prioritizing 6-&gt;8 migrations first, during the <a href="https://prague2013.drupal.org/session/drupal-8-leadership-qa.html">Drupal 8 Leadership Q&amp;A</a> session.</p> <p> Framework managers signed off on a major API change for 8.1.x to <a href="https://www.drupal.org/node/2668742">make migrations plugins instead of configuration entities</a> (now committed with a few remaining followups).</p> > Related issue: [Issue #2456259](https://www.drupal.org/node/2456259) > Related issue: [Issue #2561243](https://www.drupal.org/node/2561243) > Related issue: [Issue #2905491](https://www.drupal.org/node/2905491) > Related issue: [Issue #2859225](https://www.drupal.org/node/2859225) > Related issue: [Issue #2905736](https://www.drupal.org/node/2905736) > Related issue: [Issue #2208401](https://www.drupal.org/node/2208401) > Related issue: [Issue #3120103](https://www.drupal.org/node/3120103)