### Problem/Motivation The warning `AiContextSelector: Items cut off due to token limit: 4` ended in a bare item ID list, which on a small site reads as a count. Behind it, the stored `token_count` measured an item's guidance text alone, while each item is injected as a block with an `id`, `label` and `purpose` header and four-space indentation. The listing badge, its tooltip and the consumer budget summary compared the smaller figure against **Max tokens for injection**, so items reading 95 % of the budget could still be truncated at render time. ### Proposed resolution Store the cost of the rendered block in `token_count` (update 10032 recounts every revision and translation in chunks), compute it through the renderer's `estimateItemTokens()` / `estimateBlockTokens()` so the figure and the render cannot drift, recount on label or purpose changes and on publish or revert, and make the warning say what the number is: how many of the selected items did not fit which budget, for which consumer, and their IDs. Also make a parent row's tooltip say that its figure already covers its subcontext items, which are listed separately, and make the consumer budget summary note describe the new counting. ### Remaining tasks - [ ] Resolve the conflict with !370 (#3586661+s). The two MRs change the same four files. The MR that merges second must keep `estimateItemTokens()` in `preSave()` together with the approximate-count handling from !370, and make update 10032 catch `AiContextTokenizerUnavailableException` and defer the recount to cron, as !370 does for update 10023. ### API changes `AiContextItem::getTokenCount()` returns a larger number for the same item. `AiContextRendererInterface` gains two methods (the interface is `@internal`) that a decorator must implement. See `docs/developers/api-stability.md`. ### Release notes snippet The stored token count of each context item (`AiContextItem::getTokenCount()`) now includes the ID, label and purpose header and the indentation of its rendered block, not only the guidance text. Before this change, the badges in the **Tokens** column on `/admin/config/ai/context/items` and the consumer budget summary showed too low a figure, so items could be cut off while the summary said that they were within budget. Run `drush updatedb`, then `drush cr`. Update 10032 recounts every revision and translation. Selection and rendering do not change, but expect higher token badges, tooltip percentages and budget totals, and possibly a **Context budget exceeded** message for a consumer that is near its **Max tokens for injection**. The warning about cut-off items now gives the number of items that did not fit, the token budget, the consumer ID when the selection has a consumer, and the item IDs. ### Original report (ultimike, 2026-10-26) Using a fresh install of 1.0.0-rc1, with default values for _Max global context items_ (3) and _Max tokens for injection_ (1200), I am getting the following warning (via AI Observability and Logger DB modules) when using a very simple AI CKEditor consumer ("Generate with AI"): > AiContextSelector: Items cut off due to token limit: 4 I only have 3 Published context items defined, and together they total 1,061 tokens (according to the numbers on /admin/config/ai/context/items.) Attached are the 3 context items exported using `dr content:export`: [content.zip](/uploads/00deb615e77b84cff996b4cff40ace36/content.zip) When I increase the _Max tokens for injection_ value to 1500, the warning goes away and all three context items are used. Therefore, it appears: (1) the warning message isn't very helpful, as I have no idea what "4" refers to; (2) the token count on /admin/config/ai/context/items either isn't accurate or doesn't reflect what is actually being counted. <details><summary>Log record</summary> ``` message_raw: AiContextSelector: Items cut off due to token limit: @items @items: 4 channel: ai_context uid: 1 request_uri: https://aiccc.ddev.site/api/ai-ckeditor/request/basic_html/ai_ckeditor_completion referer: https://aiccc.ddev.site/node/add/article timestamp: 1793021806 severity: 4 ``` </details>