## Problem/motivation Our automated tests did not include tests that run a real agent, which meant CCC injection had to be verified manually every time. That is the highest-value thing to automate before release. We don't need a real AI model, because we don't care what the output is in order to see if CCC is working properly. Our job is just to see that the correct context is added based on the configuration. The `echoai` test provider from `ai_test` is enough. ## What EchoAI uniquely proves EchoAI is worth it for **injection**, not for **matching**. The scope plugin tests (for example `AiContextScopeRolesTest`) already cover NULL / FALSE / OR matching and stale values. Those tests stop at item IDs. EchoAI proves: - `AiContextSystemPromptSubscriber` actually ran - The item body is in the `ChatInput` sent to the provider - The acting account is `current_user` at agent-run time (not the node owner) - Agent config (`selection_mode`, `scope_subscriptions`, `always_include`) is applied via `AiContextRequestFactory::fromAgent()` - `ai_agents` really dispatches `AgentStartedExecutionEvent` and `BuildSystemPromptEvent` with the agent ID, tokens, and runner ID we rely on Rule of thumb: if the kernel selector already asserted IDs, EchoAI should only assert **one happy path and one leak/regression** per area. ## Current state Done. The harness is one class: `web/modules/contrib/ai_context/tests/src/Kernel/AiContextEchoAiSystemPromptTest.php` 18 tests, 686 assertions. Roles and the taxonomy / subscription ladder landed on `1.0.x` from #3586415+s and #3586394+s. The remaining selection families and the injection-path cases are in !281 (`194dd64`). GitLab `phpunit` on pipeline 933605 is green. It uses distinctive needles (`ECHO-CLIMATE`, `ECHO-EDITOR-ONLY`, …), `runAgentAndCaptureSystemPrompt()` to capture the final system prompt from `PostGenerateResponseEvent`, and asserts both that `provider_id === echoai` and that EchoAI really echoed the user message, so a silent no-op cannot pass. Do **not** add one EchoAI class per plugin. Families line up with the scope capability methods from #3586419+s — `supportsSubscriptions()`, `supportsExactMatch()`, `supportsSituationalMatch()`. | Family | Scopes | EchoAI job | Status | |--------|--------|------------|--------| | Situational | Taxonomy Terms, Languages, Site Sections, Entity Types | Positive match in Relevant; miss never appears, even in Broad | **Done.** Taxonomy ladder: `providerSelectionModePromptContents` plus `testSiteDefaultRelevantInjectsSituationalContext`. Language fail-closed: `testLanguageMissNeverAppearsEvenInBroad`. Path token: `testPathTokenMatchesSiteSectionThroughAgent`. | | Subscription-only | Use Cases, Context Tags | Subscribed value in Minimal; unscoped item stays out | **Done.** `testEmptyScopeDoesNotMatchSubscription`, `testAgentMinimalOverrideBeatsBroadSiteDefault`. Context Tags not needed — same family. | | Eligibility / audience | Roles | Gate holds in the prompt; match never injects on its own | **Done on `1.0.x`** from #3586415+s: `testRolesGatesSituationalItemByActingUser`, `testRoleMatchDoesNotInjectInRelevant`, `testBroadLeftoverFillStillRespectsRoles`, `testAlwaysIncludeBypassesRolesHardFilter` | | Exact-match | Specific Entities (`ai_context_entity_item`) | Current entity in Minimal; other entity never appears | **Done.** `testExactMatchInjectsCurrentEntityOnly` | | Global | Global | Present in every mode, up to the global cap | **Done.** `testGlobalItemAppearsInEveryModeUpToCap` — HIGH in Minimal / Relevant / Broad; `max_global_items=1` holds when Roles leftover fill runs | ## Injection path (also done) The family table is organised by scope, so it missed the composed `ai_agents` dispatch. Those cases are in !281 too: | Case | Method | Status | |------|--------|--------| | Injection toggle OFF | `testInjectionOffSuppressesAllContext` | **Done.** No context at all, including Global and Always include | | Usage tracking after a real run | `testUsageTrackingRecordsSelectedItemsAfterAgentRun` | **Done.** Item IDs, agent ID, non-empty runner ID, attached node | | Unpublished / draft never reaches the provider | `testUnpublishedItemNeverReachesProvider` | **Done.** Out of Broad and Always include | | Required child in prompt; conditional child out | `testRequiredChildAppearsAndConditionalChildDoesNot` | **Done.** Conditional AI disabled so EchoAI JSON cannot decide | | Path token → Site Section | `testPathTokenMatchesSiteSectionThroughAgent` | **Done.** `/landing/campaign` hits; a miss path does not, even in Broad | ## How it was structured Methods stay on `AiContextEchoAiSystemPromptTest` (same Climate article, same probe agent), swapping `current_user` between Editor and Contributor. Helper extensions: `allowContextInjection`, `extraTokens`, `maxGlobalItems`. The class uses `#[RunTestsInSeparateProcesses]`, so each method pays a full kernel boot. Locally the file went from about 32s (10 tests) to about 56–62s (18 tests). Extra `determineSolvability()` calls inside one method share that kernel and are cheaper than adding methods. That cost is the ticket's "real agent" contract, not a helper leak. ## Note on `testAlwaysIncludeBypassesRolesHardFilter` That test pins the current, documented behaviour: **Always include** skips hard filters, including the Roles gate. It is a leak path by design in 1.0.x. If the 2.x contract changes so Always include honours audience gating, this test has to change with it — it is an intentional-behaviour lock, not a bug guard. ## Out of scope / follow-up - Authenticated vs anonymous **in the prompt** — only if we care about public agents. - Agent `masquerade_roles` / cron delegated account — that is the delegated-account follow-up. EchoAI is the right place for it when it lands. - Pull-based retrieval through the tool plugin while injection is off. Not feasible with `echoai`, which echoes rather than calling tools; that needs a tool-calling mock provider and belongs in its own issue. - Lower priority and not done here: translated item body reaching the provider; `loop_aware` skip on loop > 0; custom `context_prefix`; empty-result byte-identical prompt. - Already covered elsewhere, not worth an EchoAI case: `max_tokens` truncation, item entity access, and per-plugin scope value matching. - Broad leftover for an exact-match miss (`FALSE` treated as `NULL`) was considered and skipped. The ticket asked for one happy path and one leak per family; Minimal plus the selector ID tests cover it. ## AI disclosure - [x] AI assisted issue