## Summary Follow-up to: - #3586337+s - #3586348+s Implement the subscription-limit UX decisions from the Emma+Aidan review on #3586337. Persist each context item's estimated token count at save time, base token badges on percentage of the configured token budget, and add budget summary messaging on the agent subscription settings page. This issue implements the indicator behavior. It does not add new warning components beyond text, badges, and tooltips agreed in the design review. **Implementation:** [MR !248](https://git.drupalcode.org/project/ai_context/-/merge_requests/248) on branch `3586379-save-token-counts` (latest `fb96921`). **Status:** Merged onto current `1.0.x` (token-count install is update **`10023`**; `10022` is Dynamic Entity Reference storage from #3586375). Ready for independent re-review. Confirm CI green on latest SHA. ## Problem Token counts on the context item listing are calculated on the fly from item content. That makes listing performance and budget calculations harder to reuse elsewhere, such as agent subscription settings. Current token badges also use fixed absolute tiers (under 1K, 1K–3K, over 3K). Those tiers do not reflect whether an item is a large share of the site's configured token budget. Site builders cannot easily tell from the listing or agent settings whether subscribed context may exceed the budget and therefore may not all be included in a prompt. ## Goal A site builder should be able to see: - How large each context item is relative to the configured token budget. - Approximately how much context an agent has subscribed to. - When subscribed context may exceed the budget, so not all applicable context will be included. ## Proposed solution **Note: From artem, we should use yethee/tiktoken like ai core** Implementation uses Drupal AI's `@ai.tokenizer` service (tiktoken under the hood). The `ai_context` module does not declare a direct Composer dependency on tiktoken. The model name only selects a local encoding table; there is no live model call. Resolved tokenizer model, in order: explicit `provider_config.model`, else the site-wide AI chat default when `use_default` is enabled, else `gpt-4`. ### 1. Persist token count on context item save Adds a translatable, revisionable `token_count` base field on `ai_context_item`. - Calculate the value on save using `AiContextTokenEstimator`. - Re-estimate only when a translation's `content` changed (status-only or label-only saves keep the stored count). - Recalculate for every translation on save. - Backfill existing items via `ai_context_update_10023()`. The update throws if the field cannot be installed, so schema never advances past a missing column. The batch skips items deleted mid-run. - When the resolved tokenizer model changes, stored counts are backfilled (batch on the General settings form, queue/cron on other web saves, synchronously on CLI). Triggers: - CCC General settings provider/model (including switching onto or off the site chat default when that changes the model name) - Direct `ai_context.settings` save and configuration import - Site-wide AI chat default in `ai.settings`, when CCC is using that default - Configuration imports record the change in state and act at `ConfigEvents::IMPORT` (UI imports span several batch requests). The listing and subscription summary read stored values rather than re-estimating from raw content on every request. ### 2. Base token badges on percentage of `max_tokens` Replace absolute tier thresholds with budget-relative thresholds: - **Gray:** under 10% of `max_tokens` - **Yellow:** 10% through 19.99% - **Red:** 20% or higher Percentage formula: `item tokens ÷ max_tokens × 100` (display uses `floor()` so tier boundaries align with tooltips). Compact badge labels show raw counts below 1,000 (e.g. `17`, `120`) and `K` notation from 1,000 upward (e.g. `2K`, `2–12K` for parent ranges). Token badge height is 24px (WCAG 2.5.8). When an item shows a token range because of conditional subcontext, base the badge on the high end of the range. ### 3. Update token badge tooltip on the context item listing Include the percentage in the tooltip, for example: > 400 tokens — 20% of the 2,000-token context budget. When a range is shown, the tooltip explains the range and uses the high-end percentage for the badge state. Tooltips use `data-ai-tooltip` (via `ai/ai_global.js`), which adds `tabindex="0"`, `aria-describedby`, and keyboard-focus popover support. ### 4. Add budget summary text to agent subscription settings Summary text at the top of the agent subscription settings form shows how much context the agent has subscribed to relative to `max_tokens`. Loaded via a lazy builder at render time; results are cached per agent config, module settings, scope-plugin configuration, and request context for one hour. Cache tags include `ai_context_item_list`, `config:ai_context.settings`, `config:ai_context.agents`, and each `config:ai_context.scope_settings.*` object. Suggested states: **Within budget** > **Context budget:** This agent subscribes to approximately 1,500 of 2,000 > available tokens (75%). The context included in an individual request > depends on which items apply. **Range (conditional subcontext)** > **Context budget:** This agent may use between 800 and 1,600 of 2,000 > available tokens, depending on which context applies to the request. **Over budget** > **Context budget exceeded:** This agent subscribes to as many as 2,500 > tokens, but the context budget is 2,000 tokens (125%). Not all applicable > context may be included in a request. Review this agent's subscriptions > and context conditions, or increase the maximum token budget in General > settings. Link **General settings** to the existing settings route. The form also notes that counts use stored guidance content only; rendered context adds metadata and separators, so runtime usage is usually higher. ## Implementation notes (MR !248) Beyond the four UX bullets above, the MR includes correctness and performance work needed for trustworthy indicators: - Renderer enforces `max_tokens` on fully assembled output and uses token-aware truncation with the same resolved tokenizer model as the estimator. - Agent budget calculator mirrors selector rules (`MINIMAL` with no subscriptions, `match_all`, conditional children, always-included conditional low bound, `never_include` children). - Calculator uses `AiContextLanguageService` for detected content language (same as the selector). - Excludes inheriting children when their parent is not in the budget candidate set (`MINIMAL` parity with selector candidates). - Narrow SQL queries and cached budget summaries for large catalogs. These do not add new UI components; they keep listing badges and agent summaries consistent with runtime behavior. ## Non-goals This issue does not: - Add new warning banners, modals, or dashboard widgets beyond the agreed text and badge treatment. - Change selection, ranking, or subscription scoring logic. - Change listing or agent indicator UX beyond persisting counts and showing budget-relative badges/summary. Renderer truncation was aligned with the tokenizer for budget accuracy (tiktoken via `@ai.tokenizer`). That supports accurate limits but is not separate indicator UX. ## Affected areas - `ai_context_item` entity definition and save hooks - `AiContextItemListBuilder` token badge rendering - `AiContextTokenEstimator` (resolved model, tiers, compact labels) - `AiContextSubscriptionBudgetCalculator` and `AiContextAgentForm` - `AiContextAgentBudgetSummaryLazyBuilder` - `AiContextTokenCountBackfillScheduler` and `AiContextSettingsConfigSubscriber` - Token badge theme/template and CSS - `docs/developers/services.md`, `docs/developers/configuration.md` (update 10023 recovery and model-change recount), `docs/developers/api-stability.md` - Kernel and functional tests (see MR description for full list) ## Acceptance criteria - [x] Context items store an estimated token count after save. - [x] Existing context items receive token counts via update/backfill. - [x] Stored counts refresh when the resolved tokenizer model changes (CCC General settings, configuration import, and site chat default when CCC uses that default). - [x] Context item listing badges use budget-relative gray/yellow/red thresholds based on `max_tokens`. - [x] Compact badge labels show raw counts below 1,000. - [x] Token badge height is 24px (WCAG 2.5.8). - [x] Token badge tooltips include the percentage of budget. - [x] Agent subscription settings show a budget summary at the top of the form. - [x] Over-budget messaging explains that not all applicable context may be included. - [x] Conditional subcontext is reflected as a range where applicable. - [x] Listing and form output remain accessible (tooltip via `data-ai-tooltip`; high tier includes a warning icon; color is not the only indicator on high tier). - [x] Tests cover persistence, tier thresholds, tooltip content, subscription summary states, and model-change recount (including `use_default` and site chat default). - [x] `./lint.sh` passes. ## Manual test plan 1. Check out the MR branch and run `drush updb -y && drush cr` so update **`10023`** backfills `token_count`. 2. Edit and save a context item. Confirm stored token count updates when content changes and stays the same when only status/label changes. 3. Open the context item listing (`/admin/config/ai/context/items`). Confirm badge colors at approximately 5%, 15%, and 25% of budget (adjust item content or `max_tokens` as needed; install default `max_tokens` is **1200**). Confirm sub-1K items show raw counts (e.g. `17`) rather than `0K`. 4. Hover or focus a token badge and confirm tooltip shows count and percentage. 5. Open agent subscription settings (`/admin/config/ai/context/settings/agents/{agent}/edit`) for an agent within budget, over budget, and with conditional subcontext range. Confirm summary text and the content-only note. 6. Confirm summary text matches the three states above. 7. Follow the General settings link from over-budget messaging. 8. Run accessibility checks on the listing and agent settings form. **Verified locally:** badge tooltips exposed via `aria-describedby` and keyboard focus; high-tier warning icon is decorative (`aria-hidden`); table caption references token count; agent budget summary precedes form fields; over-budget General settings link is keyboard reachable; token badge height is 24px. 9. Change the AI provider/model on General settings (`/admin/config/ai/context/settings/general`) and save; confirm a batch recount runs when context items exist. 10. Optional: set CCC to use the site chat default, then change the site-wide chat default under AI settings. Confirm stored counts are recounted (CLI is synchronous; a UI save queues the backfill for cron unless a batch is already running). 11. Optional: set **Conditional subcontext** off at `/admin/config/ai/context/settings/items` and confirm agent budget no longer treats conditional children as part of the high bound unless always-included. ## Post-merge Sites upgrading from before this MR must run: ```bash drush updatedb -y drush cr ``` No config import required. If the `token_count` column exists but listing badges stay at `0`, re-run the backfill without re-running the field install: ```bash drush php:eval "\Drupal::service('ai_context.token_count_backfill_scheduler')->backfillAll();" ``` Do **not** reset `system.schema` to an earlier version. Update `10023` is idempotent and throws if the field cannot be installed. See `docs/developers/configuration.md` (Update 10023 and Recounting after a model change). ## Automated tests (MR !248) ```bash cd web/modules/contrib/ai_context ./lint.sh cd ../../../.. ddev phpunit web/modules/contrib/ai_context/tests/src/Kernel/AiContextTokenEstimatorTest.php ddev phpunit web/modules/contrib/ai_context/tests/src/Kernel/AiContextUpdate10020Test.php ddev phpunit web/modules/contrib/ai_context/tests/src/Kernel/AiContextSubscriptionBudgetCalculatorTest.php ddev phpunit web/modules/contrib/ai_context/tests/src/Kernel/AiContextSubscriptionBudgetMinimalParityTest.php ddev phpunit web/modules/contrib/ai_context/tests/src/Kernel/AiContextRendererTest.php ddev phpunit web/modules/contrib/ai_context/tests/src/Kernel/AiContextSelectorEventsTest.php ddev phpunit web/modules/contrib/ai_context/tests/src/Kernel/AiContextItemTranslationTest.php --filter TokenCount ddev phpunit web/modules/contrib/ai_context/tests/src/Kernel/AiContextSettingsConfigValidationTest.php --filter Recount ddev phpunit web/modules/contrib/ai_context/tests/src/Kernel/AiContextSettingsConfigImportValidationTest.php --filter ImportRecalculates ddev phpunit web/modules/contrib/ai_context/tests/src/Functional/AiContextItemListTest.php --filter testTokensColumn ddev phpunit web/modules/contrib/ai_context/tests/src/Functional/AiContextAgentSettingsTest.php --filter testAgentContextBudgetSummary ``` The update test class is still named `AiContextUpdate10020Test`; it covers `ai_context_update_10023()`. ## Related issues - #3586337+s — subscription-limit indicator design - #3586348+s — help text and documentation for limits and selection - #3586375+s — Dynamic Entity Reference field storage (took update `10022`) ## AI usage - [x] AI assisted issue