Standardize thread/caller correlation on AI OTel spans (map tags to gen_ai.conversation.id)
## Problem Since project/ai_agents#3574487 landed, everything needed to reconstruct the full trail of a conversation (user, agent, and sub-agent calls) is present on the provider events. But the only way a consumer can get at it is by parsing magic prefixes out of the free-form `tags` string array on the span: | tag prefix | stamped by | meaning | |---|---|---| | `ai_assistant_thread_*` | `ai_assistant_api` (`AiAssistantApiRunner`) | assistant conversation, shared across turns | | `ai_agents_runner_*` | `ai_agents` (`AgentHelper`) | this agent run | | `ai_agents_caller_runner_*` | `ai_agents` since #3574487 (merged 2026-07-02) | the parent (caller) agent's runner | | `ai_agents_thread_*` | `ai_agents` since #3574487 | agent thread, identical for parent and sub-agent calls | There are also the ad-hoc span attributes `provider_request_id` and `provider_request_parent_id` (from `requestThreadId` / `requestParentId`). Every consumer that wants to show a trail has to parse these prefixes itself and rebuild the same grouping logic. That applies to a Grafana dashboard, the ai_agents_debugger UI, an OTLP importer, and any external APM. The prefixes are an undocumented convention with no owner, so consumers drift: #3574487 shipped a new prefix that no existing consumer reads yet. The OTel GenAI semantic conventions already define attributes for exactly this. `gen_ai.conversation.id` is "the unique identifier for a conversation (session, thread), used to store and correlate messages", and `gen_ai.agent.id` / `gen_ai.agent.name` identify the agent. #3586479 standardized what a call was (provider, model, tokens, finish reasons). This issue standardizes how calls relate to each other. ## Proposed approach: additive, dual-emit, same pattern as #3586479 In `AiOtelSpansEventSubscriber`, parse the well-known tag prefixes once at emission time and emit real attributes next to the raw `tags` array. The `tags` array stays, as documented legacy. | source | attribute | note | |---|---|---| | `ai_assistant_thread_<id>` tag, else `ai_agents_thread_<id>` tag | `gen_ai.conversation.id` | precedence rule: the assistant thread wins when both are present, since the assistant orchestrates the agents; one conversation id per span | | `ai_agents_runner_<id>` tag | `gen_ai.agent.id` | the agent run this call belongs to | | `ai_agents_caller_runner_<id>` tag | a documented ad-hoc attribute, e.g. `drupal.ai.caller_runner_id` | no semconv equivalent exists; candidate for real OTel span parenting later (see out of scope) | | `requestThreadId` / `requestParentId` | unchanged (`provider_request_id` / `provider_request_parent_id`) | already emitted; document them as part of this contract | Also document the contract in one place, in the ai_observability docs: which module stamps which prefix, what each one means, and which attribute it maps to. A new prefix from another module then has a place to register instead of becoming the next piece of folklore. After this, consumers read one attribute set. Grafana can group on `gen_ai.conversation.id` without custom parsing, ai_agents_debugger and importers stop regexing string arrays, and the trail scotteuser reconstructed by hand in #3574487 becomes a plain query. ## Out of scope (follow-ups, please don't expand this issue) - What `ai_assistant_api` and `ai_agents` stamp stays untouched. This issue only adds a parse-and-promote step at the OTel boundary. - Real OTel span parenting (activating span context so sub-agent spans are true children and `parent_span_id` reflects the caller). Probably the right long-term fix, but it changes trace topology for existing dashboards, so it needs its own issue. - Message content capture (`gen_ai.input.messages` / `gen_ai.output.messages`) stays deferred, same as in #3586479. ## Definition of done - `gen_ai.conversation.id`, `gen_ai.agent.id`, and the documented caller attribute are emitted next to the existing `tags` array when spans are enabled. - The precedence rule (assistant over agent thread) is implemented and tested, including the case where both tags are present. - The tag-prefix contract table is in the ai_observability documentation. - Kernel test coverage for the parse step: no tags, one tag, both thread tags, caller present. The tags themselves are aalin's and scotteuser's work in project/ai_agents#3574487. This issue just gives them a standard shape that every surface can read. --- AI-Generated: Yes (Used Claude Code to research the tag/attribute landscape and draft this issue; reviewed and edited by zorz.)
issue