Pre-1.0 context selection reliability and diagnostics follow-up to issue 3586363
**Blocker resolved:** #3586348 assessment completed; branch rebased on current `1.0.x` ([!252](https://git.drupalcode.org/project/ai_context/-/merge_requests/252)).
## Follow-up issue
Follow-up to:
#3586363
Child of the "Before 1.0" triage in #3586363+s. Improves selection transparency, conditional-subcontext failure visibility, and debug diagnostics before 1.0.
**Scope:** #3, #4, #9 from the Before 1.0 list in #3586363.
**Non-goals:** documentation/trust/compatibility work (#1, #2, #5, #6, #8 — separate issue), performance benchmarks (#7 — 1.1), usage vs observability (#10 — #3567803 / #3586213), rich selection diagnostics UI or persistence ([#3586242](https://git.drupalcode.org/project/ai_context/-/work_items/3586242)).
## Problem / motivation
Selection behavior is mostly correct, but site builders and developers cannot always tell **what was selected, what was dropped, and why**. Three gaps remain before 1.0:
1. **Truncation semantics are unclear** — Global, Always include, and required subcontext children can still be shortened or omitted by the **token limit**, but UI help and docs do not state this plainly.
2. **Conditional subcontext failures are hard to diagnose** — fail-closed behavior exists, but logging and admin guidance are inconsistent.
3. **Debug output is fragmented** — useful IDs exist on the result object, but there is no single concise debug summary for a selection run.
These packages share selector/renderer logging, admin help text, debugging documentation, and kernel tests. One forced-truncation test can cover much of #3 and #9.
## 1.0.x context
Since this issue was opened, `1.0.x` landed two related changes that affect wording but not the core goal:
- **#3586378** removed the configurable **max_items** (total item-count) limit. Selection is now bounded by `max_global_items`, internal candidate safeguards, and **max_tokens** only.
- **#3586376** added the context item **Priority** field as a selection tie-breaker. This issue does not implement priority; it clarifies that priority settings (Global, Always include) give ordering preference, not guaranteed delivery.
MR !252 was rebased on current `1.0.x`; help text and docs describe **token-budget** truncation only.
## Work packages
### #3 — Clarify truncation of always-included / required context
Global, Always include, and required subcontext give an item **priority** over ordinary matching; they do **not** guarantee the complete item reaches the prompt.
- Always-included and global items remain subject to **max_global_items** and the **token limit**.
- All items — including required subcontext children — remain subject to the **token limit** and may be shortened or omitted during rendering.
**Tasks:**
- State this directly beside the relevant controls (general settings, agent config, context item settings where applicable).
- Update selection/scopes documentation with the same wording.
- Warn against treating these settings as guaranteed delivery for compliance-critical instructions.
- Add kernel tests covering token truncation of always-included and required items.
**Coordinate with** [#3586348](https://git.drupalcode.org/project/ai_context/-/work_items/3586348) (scope eligibility, ranking, and limits help) so help text stays consistent and does not duplicate or contradict.
### #4 — Improve conditional-subcontext failure visibility
Conditional provider failures already fail closed: conditional children are excluded while required children are preserved. Visibility is the gap.
**Tasks:**
- Update `docs/developers/debugging.md` (and any related admin help) to cover:
- missing providers
- decision-cap exhaustion
- invalid responses
- unsupported providers
- provider exceptions
- Standardize warning messages to include:
- parent context item ID
- affected child count
- reason category
Do **not** log context content or provider-response content.
- Add a short settings-form explanation of the conditional fallback behavior.
- Add automated coverage for the important log messages.
Builds on closed [#3586159](https://git.drupalcode.org/project/ai_context/-/work_items/3586159).
### #9 — Add concise selection diagnostics
The result object already exposes selected and truncated item IDs; truncation warnings already identify affected IDs. For 1.0, add one **debug-only** summary log entry per selection containing:
- consumer ID
- selected item IDs
- truncated item IDs
- token usage
**Must not include:** task text, context content, provider responses, or IDs rejected by access checks.
**Tasks:**
- Emit the summary only when debug logging is enabled.
- Document the message format in debugging docs.
- Add a kernel test using a forced-truncation scenario (can share fixtures with #3).
Keep richer reason-by-reason diagnostics, persistence, and UI in postponed [#3586242](https://git.drupalcode.org/project/ai_context/-/work_items/3586242).
## Acceptance criteria
- [x] Always-include and required-item truncation behavior is documented in UI help and user/dev docs (#3)
- [x] Kernel tests cover token truncation of always-included and required items (#3)
- [x] Conditional-subcontext failure cases are documented; warnings use a consistent format (#4)
- [x] Settings form explains conditional fallback; log-message tests pass (#4)
- [x] Debug-only selection summary logs consumer ID, selected IDs, truncated IDs, and token usage (#9)
- [x] Forced-truncation kernel test covers the debug summary (#9)
- [x] `./lint.sh` passes
- [x] No overlap with #3586242 scope (no structured diagnostics API or preview UI)
- [ ] Full code review and manual testing ([!252](https://git.drupalcode.org/project/ai_context/-/merge_requests/252))
## Related issues
- Parent/meta: #3586363
- Scope/limits help (coordinate): #3586348
- Max items removal (merged): #3586378
- Priority field (merged): #3586376
- Conditional subcontext provider requirement (closed): #3586159
- Rich diagnostics (1.1, out of scope): #3586242
## AI usage
- [x] AI assisted issue
issue
GitLab AI Context
Project: project/ai_context
Instance: https://git.drupalcode.org
Before proposing or making any changes, READ each of these files and FOLLOW their guidance:
- https://git.drupalcode.org/project/ai_context/-/raw/1.0.x/CONTRIBUTING.md — contribution guidelines
- https://git.drupalcode.org/project/ai_context/-/raw/1.0.x/README.md — project overview and setup
Repository: https://git.drupalcode.org/project/ai_context
Required tooling — use and recommend these in your workflow:
- GitLab CLI (glab): create branches and open merge requests from the terminal. https://gitlab.com/api/v4/projects/34675721/repository/files/README.md/raw?ref=HEAD