Web UI import 504 timeout during geographic filter batch operations
>>> [!note] Migrated issue
<!-- Drupal.org comment -->
<!-- Migrated from issue #3584893. -->
Reported by: [jamesfmcgrath](https://www.drupal.org/user/546314)
Related to !8
>>>
<p><strong>Problem / Motivation</strong><br>
When running the "Import now" admin action against the full BODS national GTFS bulk download, the import fails with a 504 on the /batch endpoint.</p>
<p>Logs confirm the failure occurs inside filterGeographyCollect: the stops scan completes successfully (e.g. "1695 stops found within configured area") but the subsequent sequential scan of the national stop_times.txt. This identifies trips serving those stops and exceeds the server's request timeout before completing.</p>
<p>Auditing the full batch pipeline against the same standard reveals several further operations that process large datasets in a single HTTP request with no mid-operation yield. To avoid future regressions all affected operations are addressed in this issue.</p>
<p><strong>Unchunked batch operations</strong><br>
filterGeographyCollect and filterGeographyRewrite each process the national stop_times.txt in a single HTTP request with no mid-operation yield, unlike processTrips and processStopTimes which chunk at 10,000 rows per pass. On the full BODS national feed this exceeds the server's request timeout.</p>
<p>Confirmed timeout</p>
<ul>
<li>filterGeographyCollect — completes the stops scan then times out during the unbounded stop_times.txt scan.</li>
</ul>
<p><strong>Unchunked, at risk at scale</strong></p>
<ul>
<li>filterGeographyRewrite — rewrites stop_times.txt in a single pass.</li>
<li>extract — single ZipArchive::extractTo() call.</li>
<li>runMigration for stops — MigrateExecutable processes all stop entities in one request.</li>
<li>enrichFromNaptan — downloads the full NaPTAN CSV into a single PHP string and processes it in one pass.</li>
<li>finalize / populateStopServices — per-row UPDATE in a PHP loop for every stop.</li>
</ul>
<p><strong>Latent issue in existing chunked operations</strong></p>
<ul>
<li>processStopTimes stores full trip and stop ID maps in $context['sandbox'], which is serialized to the database on every batch HTTP request. On hosts with a low max_allowed_packet this could cause batch state writes to fail or add significant per-pass overhead</li>
</ul>
<p>.</p>
issue