Streamed chatbot agent response
>>> [!note] Migrated issue
<!-- Drupal.org comment -->
<!-- Migrated from issue #3578168. -->
Reported by: [d0t15t](https://www.drupal.org/user/132353)
Related to !1304
>>>
<h3 id="summary-problem-motivation">Problem/Motivation</h3>
<p>When using an <code>AiAssistant</code> that is backed by an AI agent (<code>ai_agent</code> field set), enabling streaming output has no effect. The full LLM response is always returned as a single chunk rather than being streamed token by token.</p>
<p>The root cause is in <code>AiAssistantApiRunner::process()</code>: when an agent is detected, it delegates to <code>AgentRunner::runAsAgent()</code> but does not pass <code>$this->streaming</code>. As a result, the <code>ChatInput</code> passed to the agent never has <code>setStreamedOutput(TRUE)</code> called, so the agent's internal streaming mode (<code>$this->streaming</code>) is never activated, and <code>solve()</code> returns a completed string instead of a <code>StreamedChatMessageIteratorInterface</code>.</p>
<h4 id="summary-steps-reproduce">Steps to reproduce</h4>
<ul>
<li>Enable the <code>ai_assistant_api</code> and <code>ai_agents</code> modules</li>
<li>Configure an <code>AiAssistant</code> with an AI agent set (e.g. a RAG search agent)</li>
<li>Call <code>AiAssistantApiRunner::streamedOutput(TRUE)</code> followed by <code>process()</code></li>
<li>Observe that the response is returned as a single complete <code>ChatOutput</code> with a <code>ChatMessage</code>, not as an iterable streaming iterator</li>
<li>Provider: OpenAI (gpt-4 class model with streaming support)</li>
</ul>
<h3 id="summary-proposed-resolution">Proposed resolution</h3>
<p>Two changes to <code>ai_assistant_api</code>:</p>
<p><strong><code>src/AiAssistantApiRunner.php</code></strong> — pass <code>$this->streaming</code> to <code>runAsAgent()</code>:</p>
<pre><pre>return \Drupal::service('ai_assistant_api.agent_runner')->runAsAgent(<br> $this->assistant->get('ai_agent'),<br> $this->getMessageHistory(),<br> $this->getProviderAndModel(),<br> $this->getThreadsKey(),<br> $this->getVerboseMode(),<br> $this->streaming, // added<br>);</pre></pre><p><strong><code>src/Service/AgentRunner.php</code></strong> — accept the flag, enable streaming on <code>ChatInput</code>, and return the iterator:</p>
<pre><pre>use Drupal\ai\OperationType\Chat\StreamedChatMessageIteratorInterface;<br><br>public function runAsAgent(..., bool $streaming = FALSE): ChatOutput {<br> // ...<br> $input = new ChatInput($new_messages, []);<br> if ($streaming) {<br> $input->setStreamedOutput(TRUE);<br> }<br> $agent->setChatInput($input);<br> // ...<br> $response = $agent->solve();<br><br> if ($response instanceof StreamedChatMessageIteratorInterface) {<br> return new ChatOutput($response, [], []);<br> }<br> // existing non-streaming path...<br>}</pre></pre><h3 id="summary-remaining-tasks">Remaining tasks</h3>
<ul>
<li>[ ] Add test coverage for streaming with agent-backed assistants</li>
<li>[ ] Verify behaviour when agent requires multiple LLM loops (tool calls) before the final streaming response</li>
</ul>
<h3>Optional: Other details as applicable</h3>
<p><strong>API changes:</strong> <code>AgentRunner::runAsAgent()</code> gains a new optional <code>bool $streaming = FALSE</code> parameter. Backwards compatible — existing callers are unaffected.</p>
<h3 id="summary-ai-usage">AI usage</h3>
<p>[ ] AI Assisted Issue<br><br>
This issue was generated with AI assistance, but was reviewed and refined by the creator.</p>
<p>[ ] AI Generated Code<br><br>
This code was mainly generated by an AI with human guidance, and reviewed, tested, and refined by a human.</p>
> Related issue: [Issue #3577520](https://www.drupal.org/node/3577520)
issue