## Summary Parallel PHPUnit on Drupal 10 previous-major CI fails at random when a kernel test saves or renders a context item. `preSave()` estimates tokens through the AI module tokenizer, and `AiContextRenderer::render()` configures the text chunker on every render. Both paths load vendor vocab files into the shared temp directory. Concurrent workers then throw `The map of tokens and ranks has duplicates of rank` or emit warnings that Drupal CI treats as failures. `AiContextStubTokenEstimatorTrait` already exists from #3586405+s. Only a few tests used it, and it only replaced the estimator instance. This issue puts a rebuild-safe stub in one kernel test base and moves every kernel test that touches context items onto it. Seen on `BuildSystemPromptSubscriberInjectionToggleTest` in the #3586516+s pipeline (job 12330980). The failing trace went through `preSave()`. The same encoding is loaded again through `ai.text_chunker` when the subscriber renders items, so stubbing the estimator alone leaves the race open. ## Problem `.gitlab-ci.yml` sets `_PHPUNIT_CONCURRENT: 1`. The AI tokenizer caches encodings in Drupal's temp directory with no per-process isolation. Two services reach it: - `ai_context.token_estimator`, used by `AiContextItem::preSave()` - `ai.text_chunker`, used by `AiContextRenderer::render()` Replacing the estimator instance in `setUp()` does not cover the text chunker and is lost when `KernelTestBase::enableModules()` rebuilds the container. ## Proposed solution Add `AiContextKernelTestBase` that overrides `register()` and points the tokenizer argument of both `ai_context.token_estimator` and `ai.text_chunker` at `AiContextTestTokenizer`. Because the swap is part of the container definition, it survives `enableModules()` and any other rebuild. The shared `ai.tokenizer` service stays untouched. Give `AiContextTestTokenizer` character-based `getEncodedChunks()` and `decodeChunk()` so `TextChunker::chunkText()` works without an encoder. Switch every kernel test that references context items onto the base. Keep `KernelTestBase` for tests that assert real token counts or real truncation: - `AiContextTokenEstimatorTest` - `AiContextBatchBackfillTokenCountsTest` - `AiContextSubscriptionBudgetCalculatorTest` - `AiContextSubscriptionBudgetMinimalParityTest` - `AiContextRendererTest` - `AiContextItemTranslationTest` - `AiContextSelectorEventsTest` - `AiContextSettingsConfigValidationTest` - `AiContextSettingsConfigImportValidationTest` - `AiContextUpdate10020Test` Add `AiContextKernelTestBaseTest` to prove the stub is active and survives a container rebuild. Do not change production tokenizer behaviour. ## Acceptance criteria - [ ] `AiContextKernelTestBase::register()` points both `ai_context.token_estimator` and `ai.text_chunker` at `AiContextTestTokenizer` - [ ] `AiContextTestTokenizer` supports `getEncodedChunks()` and `decodeChunk()` so the text chunker works without an encoder - [ ] All `BuildSystemPromptSubscriber*` kernel tests extend the base - [ ] Tests that already used `AiContextStubTokenEstimatorTrait` move onto the same base - [ ] All other kernel tests that reference context items extend the base - [ ] Tests that assert real token counts or truncation stay on `KernelTestBase` - [ ] A kernel test proves the stub is active and survives `enableModules()` - [ ] `tests/README.md` tells new kernel tests which base to extend - [ ] `./lint.sh` passes ## Testing instructions ``` ddev phpunit \ web/modules/contrib/ai_context/tests/src/Kernel/AiContextKernelTestBaseTest.php \ web/modules/contrib/ai_context/tests/src/Kernel/BuildSystemPromptSubscriberInjectionToggleTest.php \ web/modules/contrib/ai_context/tests/src/Kernel/BuildSystemPromptSubscriberLoopAwareTest.php \ web/modules/contrib/ai_context/tests/src/Kernel/BuildSystemPromptSubscriberIsolationTest.php \ web/modules/contrib/ai_context/tests/src/Kernel/BuildSystemPromptSubscriberRequestContextTest.php ``` Confirm the Drupal 10 previous-major phpunit job is green. ## Out of scope - Leftover consumer listing (#3586516+s) - Per-process vocab cache in production - Functional tests (same race is possible there, not seen in this pipeline) - Kernel tests that never reference context items (no tokenizer path) ## Related - #3586405+s — introduced the stub tokenizer trait - #3586516+s — leftover consumers MR that hit this flake ## AI usage - [x] AI assisted issue