run-tests.sh can fail during testDiscovery on 3rd-party modules
Problem/Motivation
With _PHPUNIT_CONCURRENT=1 to use core scripts/run-tests.sh, the test discovery process can fail for errors in 3rd-party modules, such as "class not found" or "declaration ... must be compatible with ...". However, the tests in these modules which have error are never intended to be run, so they are not actually relevent. But the job is halted, thus preventing the tests in the actual project from being run.
The problem happens when setting a value for _PHPUNIT_TESTGROUPS to run a subset of the tests. The error does not happen when running all tests in a project.
Also the error does not happen for Drupal Core 10 and 11.0 and 11.1, which use PHPUnit 10. The problems start with core version 11.2+ which use PHPUnit 11 and where run-tests.sh has been modified to use the new test discovery API.
Example working OK at core 11.1.8 and PHPUnit 10.5.56
https://git.drupalcode.org/issue/webform-3547627/-/jobs/6632376#L389
Example of fail at core 11.2.0 PHPUnit Version: 11.5.40
https://git.drupalcode.org/issue/webform-3547627/-/jobs/6632522#L389
Proposed resolution
Remaining tasks
User interface changes
API changes
Data model changes
Related issue: Issue #3549601