>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3582179. --> Reported by: [marcus_johansson](https://www.drupal.org/user/385947) Related to !1489 >>> <p>[Tracker]<br> <strong>Update Summary: </strong>[One-line status update for stakeholders]<br> <strong>Short Description: </strong>Add streaming-aware guardrails that can buffer and evaluate content in real-time using start/stop regex patterns<br> <strong>Check-in Date: </strong>MM/DD/YYYY<br> [/Tracker]</p> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Follow up from <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/ai/issues/3518963" title="Status: Closed (fixed)">#3518963: [Meta] Create the concept of Guardrail agents</a></span>.</p> <p>Currently, guardrails in the AI module only operate on complete output. When a streaming response is used, a guardrail either has to implement <code>NonStreamableGuardrailInterface</code> (which forces the entire stream to be reconstructed via <code>reconstructChatOutput()</code> before the guardrail can process it) or it receives the full <code>ChatOutput</code> after the stream has finished. In both cases, the user has already received all the streamed content before any post-generation guardrail can act on it - or the streaming experience is completely broken by waiting for the full response.</p> <p>This means there is no way to evaluate content <em>during</em> streaming. For example, if an LLM starts outputting harmful content, the user sees the entire response before a guardrail can intervene. There is also no way to buffer suspicious portions of a stream and decide whether to release or suppress them in real-time.</p> <p>What is needed is a new guardrail capability that can hook into the streaming iteration itself. The guardrail itself should be able to dynamically decide when to start buffering and when to stop - using configurable start and stop regex patterns. When a start regex matches incoming streamed content, the system begins buffering instead of yielding to the consumer. Content accumulates while streaming continues, and when the stop regex matches (or the stream ends), the guardrail evaluates the buffered content and decides whether to release, suppress, or rewrite it.</p> <p>This approach is intentionally dynamic - rather than having a single hardcoded buffer strategy in the iterator, each streaming guardrail controls its own buffering behavior. This allows different guardrails to watch for different patterns and buffer different portions of the stream independently. The current hardcoded buffer logic in <code>StreamedChatMessageIterator</code> should eventually be replaced by this guardrail-driven approach, so that all streaming content evaluation goes through the guardrail system.</p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <ul> <li>Introduce a new interface (e.g. <code>StreamableGuardrailInterface</code>) for guardrails that can operate on streaming content, with methods for providing a start regex, a stop regex, and a processing callback for buffered content.</li> <li>Extend the <code>StreamedChatMessageIterator</code> (or create a guardrail-aware decorator/wrapper) so that during iteration, it can check each chunk against registered streaming guardrails' start patterns.</li> <li>When a start regex matches, the iterator begins buffering content instead of yielding it to the consumer. Streaming continues internally, but the buffered content is held back.</li> <li>When the stop regex matches (or the stream ends), the buffered content is passed to the guardrail for evaluation. The guardrail returns a result - pass (release the buffer), stop (suppress and replace with a guardrail message), or rewrite (modify the buffered content before releasing).</li> <li>Content that does not match any start regex should continue to stream through normally with no delay.</li> <li>The existing hardcoded buffer logic in <code>StreamedChatMessageIterator</code> (e.g. <code>shouldFlush()</code>) should be migrated to use this new guardrail-driven buffering system, so that all streaming content evaluation is handled dynamically through guardrails rather than baked into the iterator. This work can however only be done when guardrails can be placed everywhere.</li> <li>Consider how multiple streaming guardrails interact if more than one has overlapping start/stop patterns.</li> <li>Determine how the <code>GuardrailsEventSubscriber</code> should register streaming guardrails with the iterator - possibly via the <code>PostGenerateResponseEvent</code> or by wrapping the iterator before it is returned to the consumer.</li> </ul> <h3 id="summary-ai-usage">AI usage (if applicable)</h3> <p>[x] AI Assisted Issue<br> This issue was generated with AI assistance, but was reviewed and refined by the creator.</p> <p>[ ] AI Assisted Code<br> This code was mainly generated by a human, with AI autocompleting or parts AI generated, but under full human supervision.</p> <p>[ ] AI Generated Code<br> This code was mainly generated by an AI with human guidance, and reviewed, tested, and refined by a human.</p> <p>[ ] Vibe Coded<br> This code was generated by an AI and has only been functionally tested.</p> <p>- <strong>This issue was created with the help of AI</strong></p>