## Summary On the agent path, the module appends its context block to the system prompt in `BuildSystemPromptEvent`. AI Agents then runs `Token::replacePlain()` on the whole system prompt with the current user as `user` data. Token text in a context item, such as `[user:mail]`, is replaced with data of the user who runs the agent and sent to the AI provider. The generic chat path sends the same text without replacement. `[user:uuid]` in item text stops the agent run with an `Error`, because AI Agents passes an `AccountProxy` as `user` data and core calls `uuid()` on it. A kernel test confirms this on `1.0.x`. AI Agents also replaces tokens in its own instructions and in default information tool output that it adds to the system prompt. When a site sets one of this module's tools (List, Load, or Get Relevant) as a default information tool, the item text in that tool output is also token-replaced, and `[user:uuid]` stops the run. This path does not go through the module's append path, so the tools need their own fix. Part of #3586615+s. Target: 1.0.0. ### Why item text must not be token-replaced AI Agents replaces tokens in the system prompt for the agent's own instructions. Site builders write those instructions and choose the tokens on purpose. CCC context is replaced only because CCC appends its block before AI Agents runs the replacement. That causes three problems: 1. **Personal data goes to the AI provider.** The replacement uses the user who runs the agent. A context author can write `[user:mail]`, and the email of every user who runs the agent is sent to the provider. The site builder did not decide this. 2. **The paths behave differently.** The same item text is replaced on the agent push path and in default information tool output, but not on the generic chat path or in normal tool calls. 3. **The budget is wrong.** CCC checks the token budget before AI Agents replaces the tokens, so a replaced token can make the text longer than the budget. The `[user:uuid]` crash is a separate AI Agents bug. AI Agents passes an `AccountProxy` as `user` data, and core calls `uuid()` on it. The same error happens when an agent's own instructions contain `[user:uuid]`, and other user tokens such as `[user:created:…]` fail in the same way. An upstream fix would stop the crash, but problems 1-3 would stay. ## Proposed resolution - Add an `@internal` token neutralizer. It puts a space on both sides of the colon of tokens, so `Token::scan()` no longer matches them. This covers every type that the token service knows (`Token::getInfo()`). Core also calls `hook_tokens()` for undeclared types, for example ECA's `[current_user:mail]`, so undeclared types that look like token types are neutralized when the first part of the name, before the next colon, has no whitespace, `/` or `@`. So a custom date format such as `[current_user:created:custom:d/m/Y]` is also broken. Some IPv6 addresses, such as `[fe80::1]`, are not changed, but other forms, such as `[fd12:3456::1]`, get the spaces. Root-level data paths such as ECA's `[current_user.mail.value]` are broken when the part before the first dot is a declared token type or `current_user`, for example `[current_user . mail.value]`. After each change, the neutralizer scans the text again, at most three times. One token can hide another from the scan: with ECA, the literal text `[_eca_root_token:x]` hid `[current_user.mail.value]` in the same text. Other text that looks like a token, such as `items[i:j]`, also gets the spaces, and so does a dotted Markdown link whose first part is a declared type, such as `[node.js](https://nodejs.org)`. Times, JSON, URLs, notes with a space after an undeclared type, such as `[note: read this]`, and other root-level tokens, such as `[2]`, are not changed. - Neutralize on every CCC path, so every caller gets the same item text. The neutralizer runs in three places: - The renderer, on the assembled text. All rendered item text comes together there, and the budget checks use that text, so the budget applies to the neutralized text, also when a token spans two fields. This covers agent push, the generic chat path, the Load and Get Relevant tools, and the public selection factory. - The selector, on text that `TEXT_RENDERED` subscribers add, and on the label, purpose, and content snippet in the selected-item data. - The List tool, on its own output, because it does not use the renderer. - A site that decorates `ai_context.selection_factory` returns the inner factory text. A site that replaces it must break tokens in its own output, because the neutralizer is internal. The Get Relevant tool returns the factory text as its output. Document this in `api-stability.md` and `consumer_api.md`. - No public API signatures change. The output of the public selection factory and of `TEXT_RENDERED` changes, and `api-stability.md` records it. To read item text with its tokens as written, load the context item entity. - Do not neutralize the **Context prefix**. It is site configuration, and sites may use tokens such as `[site:name]` there. - Add help text to the **Content** field (tokens are not replaced) and the **Context prefix** field (tokens are replaced on agent requests). - Do not add an allowlist setting in this issue. Neutralizing by default can ship first; the setting can follow in a separate issue. **Decision:** CCC neutralizes item text on every path and in the public selection API, so every caller gets the same text: default information tools, normal agent tool calls, MCP, the generic chat path, and custom code that uses the selection API. Items that document real token patterns, for example for Metatag or URL aliases, show them with a space on both sides of the colon. ### Other approaches considered - **Replace tokens in context on purpose.** Not done, because of problems 1-3 above. An allowlist setting for token types that a site wants replaced, for example `[site:name]`, can be added later as a feature. Neutralizing everything stays the default. - **Remove tokens from the text.** Not done. "Contact [user:mail] for help" would become "Contact for help", and guidance about token patterns would become useless. Nothing would show the author or the model that text is missing. Neutralized text keeps the meaning, and the model can see that a placeholder was there. - **Validate item text on save.** Not enough on its own. Entity validation runs in forms, but importers, the document loader, API saves and translations can save text without it, and items saved on rc1 sites can already contain tokens. Some items also have good reasons to show tokens. Delivery-time neutralization covers all sources. A non-blocking warning in the item form could still help authors, but it is not part of this issue. - **Neutralize only where AI Agents replaces tokens, and keep the generic chat path and the public API unchanged.** Not done. The same item text would then differ by path, the internal selector and renderer would need a flag, and custom code that adds the API text to an AI Agents prompt would still expand the tokens. - **Use a character that looks the same, for example a full-width colon.** Not done. The text would look like a real token, so an agent could copy it into configuration, and it would fail with no visible reason. The visible spaces are easier to find and correct. ### Why the word "neutralize" Drupal core does not use "neutralize". The other possible words already have a different meaning in Drupal: - **Escape** means HTML escaping, for example `Html::escape()` and `HtmlEscapedText`. `Token::replace()` uses it for token values. - **Clear** is the `Token::replace()` option that removes tokens that have no value. - **Sanitize** means making output safe for HTML or markup. - **Strip** and **remove** mean deleting text, which this issue does not do. "Neutralize" says what the change does: the text stays readable, but token replacement no longer acts on it. It is also the standard security term for this kind of fix. For example, CWE-1336 is "Improper Neutralization of Special Elements Used in a Template Engine". ## Remaining tasks - [x] Kernel: known types and undeclared token-like types (`[current_user:mail]`) are broken; JSON, numeric slices, times, URLs, notes, Markdown links, and root-level tokens such as `[2]` from an ECA-like token service are unchanged; numeric token names are handled. - [x] Kernel: repeated, adjacent, and already-neutralized tokens are handled. - [x] Kernel: the renderer checks the budget on the neutralized text (real tokenizer; the test fails when the raw text is counted). - [x] EchoAI kernel: `[user:mail]`, `[site:name]`, and `[user:uuid]` in item text reach the provider unreplaced, the run completes, and the prefix still gets `[site:name]` replaced. Mail and uuid are separate cases. - [x] EchoAI kernel: long agent item text is neutralized. The budget is tested in the renderer test. - [x] Kernel: the List, Load, and Get Relevant tools break tokens in their output, including a token across the label and the purpose (`AiContextToolTokenNeutralizationTest`). - [x] Kernel: the renderer breaks a token across the label and the purpose. The selector, the public selection factory, and the generic chat path neutralize tokens, including selected-item metadata and text that `TEXT_RENDERED` subscribers add, also when no item is selected (`AiContextToolTokenNeutralizationTest`, `AiContextPreGenerateResponseSubscriberTest`). - [x] EchoAI kernel: the Load and List tools as default information tools, with push off. `[user:mail]` does not expand to the user's email, and `[user:uuid]` does not stop the run. On unfixed code, the `uuid` case stops with `Call to undefined method AccountProxy::uuid()`. - [x] Docs: `access-boundaries.md` (Authoring trust), `api-stability.md` (upgrade note), `configuration.md` (Context prefix), `consumer_api.md` (tokens in returned text), `events.md` (TEXT_RENDERED), `function_calls.md`, `services.md`, `context_items.md`, and `rag.md`. - [x] Release note. - [x] Decide the default information tool question above, and implement it. - [x] Not needed: a push-path test for tokens in the label and the purpose. Push uses the same renderer and selector, which the tests cover. - [x] Manual check in the UI on a Drupal CMS site with ECA (agent debugger and Tools explorer), 2026-10-23. It found the first two ECA problems below. An ECA check found the third. All three are fixed. - [x] ECA: root-level tokens without a colon stay unchanged (List IDs `[2]`, one-word Markdown links). - [x] Undeclared token-like types such as `[current_user:mail]` are neutralized, also with a custom date format such as `[current_user:created:custom:d/m/Y]`. IPv6 addresses such as `[fe80::1]` are not changed. - [x] ECA root-level data paths such as `[current_user.mail.value]` are broken; `[readme.md]` is unchanged. Kernel test with an ECA-like token service, and UI re-test on 2026-10-23. - [x] ECA: the literal text `[_eca_root_token:x]` no longer hides root-level data paths such as `[current_user.mail.value]` from the neutralizer. Kernel test with an ECA-like token service; the test fails with only one pass. Checked against the real ECA 3.0.10 token service on 2026-11-01: no email in the replaced text, and a second run gives the same text. - [x] Text with no token-like parts returns early, before `Token::getInfo()` (kernel test). - [x] Help text on the Content and Context prefix fields. - [x] Docs: undeclared types, root-level tokens, IPv6 forms that change, dotted Markdown links, and tokens split across outputs (`access-boundaries.md`); token patterns for authors (`context_items.md`). - [x] Open the MR (!353). Tests and `./lint.sh` pass on the MR branch. - [ ] Upstream: the AI Agents maintainers are looking into the AI Agents part. It does not block this issue. ## Release note Drupal tokens in context item text are no longer replaced when context reaches AI. Before this change, AI Agents replaced tokens such as `[user:mail]` in item text with data of the user who ran the agent, and `[user:uuid]` stopped the agent run with an error. This happened in pushed context and in the output of the List, Load and Get Relevant tools when a site used them as default information tools. CCC now neutralizes Drupal tokens on every path, for example `[user:mail]` becomes `[user : mail]`: pushed agent and chat context, the three context tools for every caller, and the public selection API, including text that `TEXT_RENDERED` subscribers add. The token budget is checked on the neutralized text. Undeclared types that look like tokens, such as `[current_user:mail]` from ECA, and ECA data paths such as `[current_user.mail.value]` are neutralized too. Bracketed text that looks like a token, such as `[note:intro]` or `items[i:j]`, also gets the spaces, and so do some IPv6 addresses, such as `[fd12:3456::1]`, and dotted Markdown links whose first part is a token type, such as `[node.js](https://nodejs.org)`. Other bracketed text, such as times, JSON, URLs and `[2]`, is unchanged. Some forms that other modules replace are not covered, for example `[current_user]` with ECA, which becomes the ID of the user who runs the agent, and undeclared tokens whose name has whitespace, `/` or `@` before its first colon. See "Authoring trust" in `access-boundaries.md`. Tokens in the **Context prefix** setting are still replaced on the agent path. Sites that used tokens in item text on purpose must write the values into the text instead. Items that document real token patterns, for example for Metatag or URL aliases, show them with a space on both sides of the colon. Custom code that needs the item text with its tokens as written must load the context item entity. ## Environment - Found on: 1.0.0-rc1 (8a57807). - MR branch: merged with 1.0.x on 2026-11-01 (`dedb6a69`). - AI Agents: 1.3.0 (`AiAgentEntityWrapper::applyTokens()`). - ECA: 3.0.10. > [!note] > Check the one that best describes your usage, or leave all unchecked if AI was not significantly used. > > * [ ] AI **Assisted Issue** \ > _Mainly written by a human; AI used for partial generation under full human supervision._ > > * [x] AI **Generated Issue** \ > _Mainly generated by AI, reviewed and approved by a human before this issue was created._