## Summary The generic push path (`AiContextPreGenerateResponseSubscriber`) has two problems. Automators have both. The AI version controls which problem a site has. **Repeated context blocks (AI 1.5.0 and later, and the next AI 1.4 release).** An automator in base mode sends one chat request for each value of its base field. All of these requests use one provider instance (`RuleBase::prepareLlmInstance()`). `ProviderProxy::__call()` does two things with the system role of that instance: * Before `PreGenerateResponseEvent`, it copies the system role into the new `ChatInput`. * After the event, it copies the system prompt of the input back to the system role. So the second request starts with the context block of the first request. The automator makes a new `ChatInput` for each request, so the input has no "pushed" marker, and the subscriber adds one more block. Request k has k blocks and can use k × `max_tokens`. `docs/developers/events.md` says that the per-instance `max_tokens` setting is the budget control. For a field with three values, the requests send 1 + 2 + 3 = 6 blocks instead of 3. **Lost context (AI 1.4.0 to 1.4.11).** `composer.json` allows `drupal/ai` `^1.4 || ^2`. AI 1.4.11 and earlier copy the system role into the input before the event, but they do not copy the changed system prompt back after the event. OpenAI-based providers (`OpenAiBasedProviderClientBase::chat()`) send only the system role. So context that the generic path pushes does not get to these providers. This applies to automators and to all other consumer types that use `PreGenerateResponseEvent`. When usage tracking is on, the module still records usage. The result status is `pushed`. **The next AI 1.4 release.** Commit `cbe279c6` on the AI 1.4.x branch (project/ai#3586825+s) adds the same copy-back as AI 1.5 in `src/Plugin/ProviderProxy.php`. It also adds the `ai_automator:id:{id}` request tag and `entity_context`. So AI 1.4.12 fixes the lost context, but then it also adds the repeated blocks. AI 1.4.12 has no release date yet. ## Steps to reproduce Repeated blocks, on AI 1.5.0 or later: 1. Install AI Automators. Create an automator in base mode, for example `llm_text_long`, whose base field has three values. 2. Turn on push for the consumer `automator:{id}`. Add one published Global context item. 3. Save the entity. 4. Look at the system prompt of each provider request. The first request has one context block, the second request has two, and the third request has three. Lost context, on AI 1.4.11 or earlier: 1. Use an OpenAI-based provider. Create an automator on a field that has no other automator. AI 1.4.11 sends no `ai_automator:id:{id}` tag, so CCC can identify the automator only when it is the only automator on the field. 2. Turn on push for the consumer `automator:{id}`. Add one published Global context item. Turn on usage tracking and debug logging. 3. Save the entity. 4. The provider request has no context block. The Usage tab records the item, and the log shows `status=pushed`. Confirmed on `1.0.x` at `65f20b72`: the subscriber adds a block and does not remove earlier blocks. The kernel tests that send requests through `ProviderProxy` fail on `1.0.x` (`9932f4ab`) with AI 1.5.1. A manual check on `9932f4ab` used a real provider (Quant Cloud, MiniMax M2.5), an automator in base mode (`llm_text_long`) on a string field with three values, and one Always include item. The three system prompts had 1, 2, and 3 context blocks (7,766, 15,532, and 23,298 characters). The same check with an earlier local fix, which used the same removal code as the current fix, gave one block for each request (7,766 characters). The lost context on AI 1.4 is confirmed by reading `ProviderProxy` at the AI 1.4.11 tag. It was not run on AI 1.4. ## Proposed resolution 1. On a routed request whose system prompt ends with the delimiter line, remove the text from the start of the first context block of this module to the end of the system prompt, before the selection. This also removes text between two blocks, because that text came from an earlier request too. Then add one block if the request pushes. A block starts with two newlines, the context prefix, a newline, and the delimiter line. It ends with the delimiter line. The removal also runs when the request does not push, for example when the status is `push_disabled`, `declined`, or `no_items`. The reentrancy check for a tool-call inner request still runs first. Requests that the subscriber does not route, for example agent requests, do not change. If another `PreGenerateResponseEvent` subscriber adds text after the block, the prompt does not end with the delimiter line, so the removal does not run. This step is necessary on AI 1.5.0 and later, and on AI 1.4.12. 2. Keep AI 1.4 support. #3586712+s changes the requirement to `~1.4.11 || ^1.5.2 || ^2`. After AI 1.4.12 is released, confirm that it has the code from `cbe279c6`. Then change `~1.4.11` to `~1.4.12`, so that the requirement is `~1.4.12 || ^1.5.2 || ^2`. With this minimum, every allowed AI release copies the system prompt back, and sends the ID tag and `entity_context`. #3586463+s planned the 1.4.12 minimum. This issue makes that change. The removal of the bridge subscribers stays in #3586463+s. 3. AI 1.5.0 and 1.5.1 have the same `ProviderProxy` code as AI 1.5.2. So for this bug, AI 1.5.0 is enough. The 1.5.2 minimum comes from #3586705+s. This issue does not change it. 4. Change the docs from #3586679+s. They must say that full Automator support needs AI 1.4.12, or AI 1.5.2 or later. Remove the "On AI 1.4.x" limits, because no allowed release has them. 5. If AI 1.4.12 is not released before 1.0.0, keep `~1.4.11 || ^1.5.2 || ^2` from #3586712+s. 1.0.0-rc2 does not wait for AI 1.4.12: its release notes list the lost context on AI 1.4.11 and earlier as a known issue. Then this issue does not change `composer.json`. The docs say that on AI 1.4.11, context that the generic path pushes (for automators and other consumer types) does not get to OpenAI-based providers, and that the result status and the usage still say `pushed`. Keep the "On AI 1.4.x" limits for AI 1.4.11. 6. Report to project/ai that the automator `RuleBase` must reset the system role for each request. This module cannot fix one case. Automators send no system prompt of their own, and `ProviderProxy` does not copy an empty prompt back to the role. So when a later request on the same provider instance does not push (for example, Relevant mode selects no item for the second value), the provider still sends the block of the earlier request. 7. Add kernel tests that send the requests through `ProviderProxy` with the EchoAI provider. The current tests dispatch the event directly with one `ChatInput`, so they cannot show this problem. Also add kernel tests that dispatch the event directly for text between two blocks, for routed requests that do not push, and for requests that the subscriber does not route. Docs: * `docs/developers/events.md`, `docs/developers/consumer_api.md`, and `docs/developers/api-stability.md` describe the new step. * `docs/features/consumer_configuration.md` and `docs/developers/debugging.md` describe the limit from step 6 until project/ai fixes it. A later request that does not push can still send the earlier block to OpenAI-based providers, while its result status says `no_items` or `push_disabled`. Add the project/ai issue number when it exists. * `README.md`, `docs/index.md`, `docs/upgrade-policy.md`, `docs/features/consumer_configuration.md`, `docs/developers/consumer_api.md`, `docs/developers/debugging.md`, and `docs/developers/events.md` get the new AI minimum, in the later commit for step 2 (or the text for step 5). ## Remaining tasks The fix, the kernel tests, and the docs are done locally, on a branch from `1.0.x` at `65f20b72`. The branch does not change `composer.json`. The change to the AI minimum comes in a later commit, after the AI 1.4.12 release. * [x] Remove earlier context blocks on routed requests. * [x] Kernel tests through `ProviderProxy` with EchoAI. They fail on `1.0.x`. * [x] Kernel tests that dispatch the event directly: text between two blocks, routed requests that do not push, and requests that the subscriber does not route. * [x] Manual check with a real provider on AI 1.5.1. * [x] Do not port the `^1.5 || ^2` requirement and the "needs AI 1.5" text from the first local fix. The change to the AI minimum text stays in the AI 1.4.12 task below. * [x] Change "AI core 1.5 and later" in the code comment, the test, and the docs so that it also names AI 1.4.12. * [x] Document the limit from step 6. * [x] ~~Rebase on `1.0.x` after the MR of #3586705+s is merged.~~ Not necessary: #3586712+s (!395) now sets the requirement, and !394 does not change `composer.json`. * [x] Moved to #3586724+s: the AI 1.4.12 check and the requirement and docs change (or the fallback), the kernel test run on AI 1.4.12, the project/ai report, and the update of #3586463+s. * [x] Open the MR: !394. It closes this issue. The AI 1.4 tasks are in #3586724+s. * [ ] Review. ## User interface changes None. ## API changes None in the PHP API. For AI 1.4, the `drupal/ai` minimum changes from 1.4.11 (set in #3586705+s) to 1.4.12. The minimum for AI 1.5 and later stays 1.5.2. With the fallback (step 5), the requirement does not change. The branch does not change the requirement yet. That change comes in a later commit, after the AI 1.4.12 release. ## Data model changes None. ## Release note Automator requests now get one context block each. Before this change, each value of a multi-value base field added one more block to the next request. CCC now requires AI 1.4.12 or later on the 1.4 branch, or AI 1.5.2 or later (`~1.4.12 || ^1.5.2 || ^2`). On AI 1.4.11 and earlier, context that CCC pushed into automator requests and other generic chat requests did not get to OpenAI-based providers. Known limit: when a later request of the same automator does not push context, OpenAI-based providers can still send the context block of the earlier request. This needs a fix in AI Automators. See "Known limit" in the Consumer configuration docs. > [!note] > Check the one that best describes your usage, or leave all unchecked if AI was not significantly used. > > * [ ] AI **Assisted Issue** \ > _Mainly written by a human; AI used for partial generation under full human supervision._ > > * [x] AI **Generated Issue** \ > _Mainly generated by AI, reviewed and approved by a human before this issue was created._