The Drupal 10 PHPUnit job never runs in CI, so Drupal 10 regressions go unnoticed

Problem/Motivation

.gitlab-ci.yml sets OPT_IN_TEST_PREVIOUS_MAJOR: 1, and pins PHP 8.3 for the previous-major jobs. That only makes the jobs available. The GitLab templates run a variant job automatically only when _AUTORUN_PREVIOUS_MAJOR asks for it, and its default is none, so composer (previous major) is created as a manual job and phpunit (previous major), which needs it, is skipped.

In the recent history of project/tool nobody has started it: the 1.0.x push pipelines for 689add5c, 18c5491d, 9e0b44d1, 8c855101 and 16199734, and the MR pipelines for !171 (merged), !181 (merged) and !182 (merged), all show composer (previous major) as manual and phpunit (previous major) as skipped. No previous-major PHPUnit job ran in the last 800 jobs of the project (back to 2026-09-24).

The cost is real. Running the kernel suite of 1.0.x on Drupal 10.6.18 (PHP 8.4, PHPUnit 9.6) shows that plain map inputs and outputs are advertised as strings there (#3583071 (closed)). AdvertisedSchemaAcceptanceTest and AdvertisedOutputSchemaConformanceTest already fail on it; CI never ran them on Drupal 10.

Steps to reproduce

  1. Open any recent pipeline of project/tool, on 1.0.x or on an MR.
  2. Observed: composer (previous major) is "manual", phpunit (previous major) and phpstan (previous major) are "skipped".

Proposed resolution

Set the autorun variable next to the opt-in in .gitlab-ci.yml:

variables:
  OPT_IN_TEST_PREVIOUS_MAJOR: 1
  _AUTORUN_PREVIOUS_MAJOR: 'push-and-mr'

The job then runs on 1.0.x pushes and on MRs. 'push-and-schedule' with a weekly schedule is a cheaper option if MR pipelines should stay short.

The job will not be green on the first run. The kernel and unit suites of 1.0.x on Drupal 10.6.18 with PHPUnit 9.6 fail in these groups (Drush command failures left out, because Drush was not installed in that local run):

  • Plain map definitions advertised as strings: AdvertisedSchemaAcceptanceTest (2), AdvertisedOutputSchemaConformanceTest (1). #3583071 (closed), fixed in !191 (merged).
  • PHPUnit 9 compatibility, in the kernel and unit suites:
    • RequiredEmptyListTest, ToolRunCommandTest and DefinitionParentageTest (unit) declare data providers only with the #[DataProvider] attribute, which PHPUnit 9 ignores ("Too few arguments").
    • Tests that expect a deprecation with #[IgnoreDeprecations] need @group legacy on PHPUnit 9 ("Only tests with the "@group legacy" annotation can expect a deprecation"): MultipleListSchemaTest, TypedInputsValidationTest, InputDefinitionRefinerTest, RefinerContractTest, and the unit tests ListInputDefinitionFromMultipleTest, ListOutputDefinitionFromMultipleTest, MapMultiplePropertyTest and OutputDefinitionCreateTest.
    • DefinitionParentageTest calls expectUserDeprecationMessage(), which PHPUnit 9 does not have.
  • Behavior differences to triage: TypedInputsValidationTest::testValidatesStructureReadingMapConstraints and ::testValidatesMapDefinitionConstraints get an extra property violation on Drupal 10; TypedInputsValidationTest::testDeclaredMapConstraintsSkipEntityValidation queries the users table, which the test does not install; EntityInputTypeCheckTest::testDeclaredConstraintsAreChecked reports one violation where none is expected.

Remaining tasks

  • Add the autorun variable.
  • Fix or explicitly skip each failing group, in this issue or in follow-ups, until the job passes.
  • Decide whether phpunit (previous major) is allowed to fail. It should not be, once it is green.

AI usage (if applicable)

  • AI Assisted Issue: This issue was generated with AI assistance, but was reviewed and refined by the creator.
  • AI Assisted Code: This code was mainly generated by a human, with AI autocompleting or parts AI generated, but under full human supervision.
  • AI Generated Code: This code was mainly generated by an AI with human guidance, and reviewed, tested, and refined by a human.
  • Vibe Coded: This code was generated by an AI and has only been functionally tested.