feat: #3579665 collect AI provider calls

🤖 This was written by an AI agent on behalf of @lussoluca.

Fixes #3579665 (closed).

What changes

AiDataCollector is registered in WebprofilerServiceProvider only when the ai module is enabled, next to the existing conditional registrations for Blocks, Views, Monolog and Symfony Messenger. It is tagged both data_collector and event_subscriber, and ai is added to the default active_toolbar_items.

The collector subscribes to the four events dispatched by Drupal\ai\Plugin\ProviderProxy, all at priority -1000 so that guardrails, failover and any other third party manipulation of input, output and configuration are already applied when the data is read:

  • ai.pre_generate_response opens the call.
  • ai.post_generate_response closes it, or leaves it open when the response is a stream.
  • ai.post_streaming_response closes a streamed call, which is the only point where its token usage exists.
  • Drupal\ai\Event\AiExceptionEvent records the failure, and marks the call recovered when a subscriber returned a replacement output.

Events are subscribed by name and their payload is read with method_exists(), so no Drupal\ai\* class is referenced. Webprofiler still installs and runs with no AI module present.

A call left open at collect time is reported rather than dropped: short_circuited when a guardrail forced an output and the provider was never called, unconsumed when a streamed response was never iterated.

Calls made on behalf of another call, by an agent or by a model backed guardrail, carry a parent request id. The collector resolves the nesting depth (guarding against cycles from reused ids) and marks nested calls with in the panel.

What the panel shows

Four tabs, modelled on the Symfony AI profiler panel:

  • Metrics. Call and failure counts, total time, token totals broken down by kind, guardrail counts, tool call count, and calls per provider, per model and per operation type.
  • Calls. One block per call: status, provider, requested model, model that answered, operation type, duration, streamed flag, thread and parent ids, tags, finish reasons, token usage, rate limits, the error when there is one, the messages with role, text, files and tool calls, the system prompt, and the raw input, output, configuration, metadata and debug data.
  • Guardrails. Every result recorded on the input, with the call it belongs to, its label, the mode it ran in, its type, whether it asked to stop, its score, its message and its context.
  • Tools. Tools offered to the provider with their parameters, and the tool calls the provider asked for with their id and arguments.

The toolbar item shows the call count and the total provider time, turns red on a failed call and yellow when a guardrail asked to stop, and stays hidden when the request made no AI call.

Serialization

PanelTrait (through DumpTrait) caches the VarCloner it builds on the collector itself, and a cloner holds closures. Any collector that clones during collect() therefore dies at kernel terminate with Serialization of 'Closure' is not allowed, the same failure fixed for EventsDataCollector in #3579661 (closed).

This collector clones while the request is running, so it cannot use the trait method. A private snapshot() delegates to DataCollector::cloneVar(), which builds a throwaway cloner. The trait method is not used anywhere in the class, at collect time or at render time, so the collector serializes at any point in its life.

Verification

On Drupal 11.4, PHP 8.4, with ai, ai_provider_openai and webprofiler enabled, driving the collector with synthesized events:

Check Result
Chat call with tools, usage and finish reason 1 call, 42 tokens (30 in, 12 out), status completed, response model and finish reason read from the raw output
Guardrail results (pass plus stop with score) 2 results collected, 1 counted as stopping, call marked short_circuited
Failed call with AiRateLimitException status error, exception class and message recorded
Nested call with a parent request id depth 1
serialize() right after collect() 7.4 KB, no closure error
unserialize() then render the panel totals and panel identical to the live collector
Twig template compiles @webprofiler/Collector/ai.html.twig loads

Two things are not covered: a real streamed response, which needs a live provider, and a browser check of the toolbar item.

No test is added. No data collector in the module has one, and a kernel test would make the AI module a test dependency of webprofiler.

Merge request reports

Loading
Loading