Issue #3616973: Run the test suite as two jobs, and stop the tests paying for installs nothing uses

Two changes, in the order they matter.

The job was bound by its cores, not by its work

The suite runs through run-tests.sh --concurrency 8, and the runner pod is scheduled with a CPU request of 2. Eight workers, two cores.

That is why taking work out stopped helping. With the browser tests already cut by a third and the longest class halved, the job still finished in the same time as a branch carrying none of it, on a runner the lint jobs showed to be the same speed: 316s against 303s.

So the job runs as two jobs, split by test type, each on a pod of its own: the kernel tests, which are about seven tenths of the work, and everything else beside them.

  • The types listed cover every type run-tests.sh knows, so a test cannot fall between the jobs. A Build or Unit-Component test added later joins the second job rather than being silently skipped.
  • Verified by counting, not by the badge: 1418 tests and 21571 assertions across the two jobs, identical to what the single job ran.
  • Two jobs rather than three. A pod costs about a minute before it runs a test, and the functional and unit tests are some twenty seconds of work between them, so they ride with the browser tests, whose pod waits on one long class rather than on its threads. Folding them in cost nothing measurable and gives a runner back.

The same split is in project_browser, which is where the pattern comes from.

About nine minutes, to about three. Single job 537s, then 316s, then two pods at 254s and 151s. Treat that as "about three minutes" rather than a figure: the kernel job, unchanged between two runs, measured 192s and 254s, so this job's variance is wide.

The tests stop paying for installs nothing uses

Real work, and it makes the second pod possible, but on its own it did not move the wall.

  • 553 entity schema installs removed across 140 kernel classes. Which ones were unnecessary was decided by running each class without them, not by reading it. Classes that count queries keep everything they declare: a test that warms a cache and then asserts nothing more is read may reach a table on one run and not the next, which is how VenueMapVersionTest passed locally and failed on CI.
  • 17 browser classes grouped, 138 tests into 28, so a site install is paid per group rather than per test. A class's setUp() body is the fixture its tests depend on; the install is the only granularity the framework offers, not the isolation they use.

RunsEveryScenario keeps what merging would otherwise cost: a scenario's failure is recorded rather than thrown, so the ones after it still run and the test fails once naming every scenario that broke. StartsFromAnEmptySite hands back the empty site to classes that read their fixture off a listing page.

Coverage

929 test bodies before, 929 after, every one unchanged. The assertion count falls only where a fixture check stops repeating: parent::setUp() asserts once that the install succeeded, drupalLogin() four times that the sign-in worked, installEntitySchema() once per table. Every class reconciles exactly against how many installs it no longer pays for.

Isolation is weaker and a scenario can no longer be selected with --filter. Those costs, and two others, are set out in a comment on the issue.

Nine browser classes were tried and left alone: four build something the site holds once, and five read their fixture off a collection page where an earlier scenario's copy would be listed beside it.

Edited by Frank Mably

Merge request reports

Loading
Loading