Fiber branch in OpenAiBasedProviderClientBase::chat() drops token usage from the reconstructed ChatOutput
## Problem/Motivation `OpenAiBasedProviderClientBase::chat()` loses all token usage data when it runs inside a PHP Fiber. The Fiber branch (line ~341 on `1.x`) consumes the stream, then reconstructs the final output: ```php // Create the final message from accumulated data. $message = $stream->reconstructChatOutput()->getNormalized(); $chat_output = new ChatOutput($message, $response, []); ``` `reconstructChatOutput()` returns a `ChatOutput` that already carries the correct token usage — but only `getNormalized()` is kept, and the new `ChatOutput` is built with an empty `TokenUsageDto`. The synchronous branch right below handles this correctly via `setChatTokenUsage()`; the Fiber branch never does. This is not an edge case: since core 11.x, `Renderer::executeInRenderContext()` wraps controller execution in a `\Fiber` (via `EarlyRenderingControllerWrapperSubscriber`), so any non-streamed chat call that runs during regular controller execution — page controllers, non-AJAX form submits, **batch operations** — hits this branch and returns NULL token usage (any provider extending `OpenAiBasedProviderClientBase` with streaming capability: openai, mistral, groq, …). Note: AJAX callbacks (e.g. the AI API Explorer form) run *outside* the Fiber wrap and are not affected, which makes the bug look intermittent. The same call via Drush also returns correct usage. Found while testing [ai_metering](https://www.drupal.org/project/ai_metering), which listens to `PostGenerateResponseEvent` to track AI usage/costs: calls made in Fiber context came in with no token data, so cost tracking silently fell back to local estimation instead of recording the real provider counts. ## Steps to reproduce 1. Drupal 11.3.x, ai 1.x (also present in 1.4.2), any OpenAI-based provider (verified with ai_provider_ollama). 2. Run a non-streamed chat call inside `Renderer::executeInRenderContext()` — the exact wrap every controller execution gets — and inspect `$output->getTokenUsage()`: all `TokenUsageDto` fields are NULL. 3. Run the identical call without the wrap (plain `drush php:script`): usage is populated (e.g. input=31, output=5, total=36 with llama3.2:3b). 4. Real-world triggers: batch operations (where this was first observed), entity save on a non-AJAX form submit, any custom controller calling `chat()` directly. ## Proposed resolution Keep the reconstructed `ChatOutput` and carry its token usage over: ```php $reconstructed = $stream->reconstructChatOutput(); $message = $reconstructed->getNormalized(); $chat_output = new ChatOutput($message, $response, [], $reconstructed->getTokenUsage()); ``` MR with a regression test (runs `chat()` inside a real `\Fiber` with a stubbed client) is up. Fix verified on a real Drupal 11.3.11 site: before, usage NULL in Fiber context; after, identical token counts in Fiber and CLI contexts. ## Related issues - #3586457 is complementary, not a duplicate: it changes *when* the Fiber branch engages; this fixes the data loss *whenever* it does. Adjacent lines, whichever lands second needs a trivial rebase. - #3545602 / #3541473 added the usage handling that the Fiber branch (#3538027) never received. Side note: on this same path `PostStreamingResponseEvent` fires with correct usage but an empty `provider_id` (`ProviderProxy::attachStreamMetadata()` never runs). Separate issue if confirmed.
issue