## Problem CCC has two overlapping concepts today: **1. Usage tracking (`ai_context_usage`)** — shipped in [#3550034](https://git.drupalcode.org/project/ai_context/-/work_items/3550034) - Lightweight Drupal-native reverse index - Answers: “Which context items were used, by which agent/run, on which route, touching which entities?” - Supports entity-update workflows when context changes - Disabled by default; capped and pruned via cron **2. AI Observability** (AI module) - Request-level audit trail and aggregated usage reports - See [#3535260](https://www.drupal.org/project/ai/issues/3535260), [#3567892](https://www.drupal.org/project/ai/issues/3567892) - Technical hook (from @murz): attach context item IDs to observability metadata on each AI request, then aggregate reports by that field Docs currently say observability **will replace** usage tracking (`docs/features/usage_tracking.md`). That may be wrong. We need an explicit product/architecture decision before 1.0 sites accumulate usage data under ambiguous expectations. ## Decision to record Choose and document **one** model: ### Option A — Dual-track (recommended default) - **`ai_context_usage` stays** as the lightweight reverse index for “which entities may need updates when context changes?” - **Observability adds** richer per-request logging and aggregated reports - **Correlate** via `runner_id` and context item IDs in observability metadata - Usage remains **disabled by default** until observability integration is clearer; describe as complementary, not “being replaced” ### Option B — Observability-primary - New usage/analytics data goes to observability only - `ai_context_usage` is **deprecated** on a defined timeline - Document migration, export, or purge policy for existing records ### Option C — Transitional hold - Usage stays as-is and default-off until a defined **1.1 observability integration** ships - Revisit A vs B at that milestone with a written migration plan ## Questions this issue must answer 1. **Long-term role of `ai_context_usage`** — permanent index, transitional, or deprecated? 2. **Responsibility split** — which questions does each system own? | Question | Usage entity | Observability | |---|---|---| | Which entities used context item X? | Primary today | Maybe via aggregation | | Per-request audit trail | No | Yes | | “How often is item X selected?” | Partial (Views) | Yes (aggregated reports) | | Tools/entities modified on a run | Yes (`runner_id`, etc.) | Via request metadata | 3. **Field correlation** — for each usage field, document keep / duplicate in observability / drop later: - `context_item_id`, agent, `runner_id`, route, target entity, tools, `changed` 4. **Existing data policy** — when observability integration lands: keep in parallel, migrate, export-only, or purge? 5. **Doc wording** — replace “will replace” in `usage_tracking.md` and settings/help text with the chosen relationship ## Pre-1.0 deliverables (decision + docs only) - [ ] Decision recorded on this issue (A, B, or C) - [ ] Short “Usage vs Observability” section added to docs (or `usage_tracking.md` updated) - [ ] #3586213 re-scoped with a comment linking back here (what stays in CCC usage UI vs moves to observability) - [ ] No new child issue unless **1.1 code integration** is explicitly scoped (wire context IDs into observability metadata, etc.) **Not in scope for this issue:** implementing #3586213 UX, selection audit trail ([#3586165](https://git.drupalcode.org/project/ai_context/-/work_items/3586165)), or opt-in selection diagnostics ([#3586242](https://git.drupalcode.org/project/ai_context/-/work_items/3586242)). ## After the decision — what to update on #3586213 Add a comment on [#3586213](https://git.drupalcode.org/project/ai_context/-/work_items/3586213): - **Blocked by:** this issue (#3567803) - Re-tag each UX item by data source once decided: | #3586213 item | Likely owner | |---|---| | Per-item token size | [#3586379](https://git.drupalcode.org/project/ai_context/-/work_items/3586379) (CCC UI), not this track | | Usage counts on context items | Depends on decision (usage entity vs observability aggregates) | | Usage over time table/chart | Likely observability reports | | Logging screen review | Observability / AI usage reports | | Live token usage in chat | Future AI Agents UX; out of scope | - Drop “rc1 if feasible” timing until architecture is settled - Milestone: 1.1 (or remain postponed) ## Related issues - Related issue: #3586363 (#10) - Usage implementation (closed): #3550034 - Usage/logging UX (re-scope after decision): #3586213 - Selection observability / audit (separate 1.1 track): #3586165 - AI module: #3535260, #3567892 ## Remaining tasks - [ ] Discuss and pick A, B, or C - [ ] Document the decision and correlation/migration policy - [ ] Update `docs/features/usage_tracking.md` and any settings/help wording - [ ] Comment on #3586213 with re-scoped backlog - [ ] Optionally spawn one 1.1 **implementation** issue for observability wiring (only if needed) ## AI usage - [x] AI assisted issue