Admin import times out: extractAndFilter batch step exceeds web server request timeout
>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3584479. --> Reported by: [jamesfmcgrath](https://www.drupal.org/user/546314) >>> <p><strong>Problem</strong></p> <p>The "Import now" admin button can time out on standard hosting environments. The extractAndFilter batch operation does too much work in a single HTTP request: it unzips the national BODS file, scans stop_times.txt twice to geo-filter the data, rewrites all five GTFS files, and copies them to the staging directory. On hosts with a typical 30&ndash;60 second request timeout the filter completes but the final file copy never finishes before the request is killed.</p> <p>Additionally, the processStopTimes initialisation pass reads the entire stop_times.txt just to count rows before doing any real work, adding unnecessary overhead to that first batch request.</p> <p><strong>Proposed resolution</strong></p> <ol> <li>Split extractAndFilter into two batch operations: extraction and filtering in one pass, merging files to staging in a second pass.</li> <li>Remove the row-counting loop from processStopTimes initialisation and derive progress from bytes read instead.</li> <li>Replace per-row UPDATE loops in populateRouteHeadsigns() and NaptanImportService::parseAndUpdate() with bulk updates.</li> </ol>
issue