feat: #3586479 Add gen_ai.* semantic conventions to ai_observability

George Kastanisrequested to merge
issue/ai-3586479:3586479-layer-2-align-ai-observability-otel-emission-to-genai-semantic-conventions-gen-ai into 1.x

Summary

ai_observability now emits the OpenTelemetry GenAI semantic-convention attributes (gen_ai.*) next to the existing ad-hoc attributes and counters. The old names are kept so current dashboards keep working. This covers the non-streaming path. Streaming finish reasons depend on #3586473 (closed) and come in a follow-up.

Spans: gen_ai.provider.name, gen_ai.operation.name (chat, text_completion, embeddings only), gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens (explicit 0 kept), gen_ai.response.model, and gen_ai.response.finish_reasons when the provider returns them.

Metrics: the gen_ai.client.token.usage histogram for input and output, with gen_ai.token.type, gen_ai.provider.name and gen_ai.request.model dimensions.

GenAiAttributeMapper maps provider and operation ids to the well-known values and falls back to the raw id. README and the docs page describe the schema and mark the ad-hoc names as legacy (removal targeted 2.0.x).

Closes #3586479 (closed)

Testing

Unit: vendor/bin/phpunit modules/ai_observability/tests/src/Unit

Verified against a live provider: a real chat through OpenRouter (gpt-4o-mini) produced a span with gen_ai.provider.name=openai, gen_ai.operation.name=chat, gen_ai.request.model, gen_ai.usage.input/output_tokens, gen_ai.response.model and gen_ai.response.finish_reasons=["stop"], next to the legacy attributes, plus the gen_ai.client.token.usage histogram. Confirmed in the exported OTLP payload.

AI Compliance

AI-Generated: Yes (Used Claude Code to write the mapper, the span and metric emission, the unit tests and the docs. Reviewed and verified by the author, including a live end-to-end run, before submission.)

Merge request reports