>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3584386. --> Reported by: [marcus_johansson](https://www.drupal.org/user/385947) >>> <p>[Tracker]<br> <strong>Update Summary: </strong>[One-line status update for stakeholders]<br> <strong>Short Description: </strong>Should the AI module's test suite install major provider modules (OpenAI, Anthropic, Ollama, Huggingface) and kernel-test that plugins still load, to catch cross-module WSOD regressions earlier?<br> <strong>Check-in Date: </strong>MM/DD/YYYY<br> [/Tracker]</p> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Today the AI module's own tests do not install the major provider modules. That means a change in the AI module can pass CI, get merged, and only later surface as a WSOD or plugin discovery failure when someone installs it together with a provider like OpenAI, Anthropic, Ollama or Huggingface. Issue <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/ai/issues/3584344" title="Status: Closed (fixed)">#3584344: The handleApiException signature change from Exception to Throwable breaks downstream providers (e.g. Anthropic) that override this method.</a></span> is a recent example of the kind of cross-module breakage this setup does not catch.</p> <p>The question for discussion is whether we should extend the AI module's test suite to install a curated set of major providers and use kernel tests to verify that plugins still load cleanly when they are present. This would give us an early signal for integration-level regressions, at the cost of coupling AI module CI to the health of the provider modules.</p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <ul> <li>Add a kernel test (or test trait) in the AI module that installs OpenAI, Anthropic, Ollama and Huggingface providers alongside AI.</li> <li>Assert that the container builds, plugin discovery runs, and each provider's plugins are loadable - no WSOD, no plugin exceptions.</li> <li>Decide which provider set is in scope. Starting with the four above keeps the blast radius small; we can expand later.</li> <li>Decide how to handle failures in a provider module - should a broken provider block AI module merges, or should the test be advisory / allowed to fail?</li> <li>Consider pinning provider module versions used in CI so that an unrelated provider release cannot unexpectedly break AI module CI.</li> </ul> <p>Trade-offs to weigh:</p> <ul> <li><strong>Advantage:</strong> Catches integration regressions like <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/ai/issues/3584344" title="Status: Closed (fixed)">#3584344: The handleApiException signature change from Exception to Throwable breaks downstream providers (e.g. Anthropic) that override this method.</a></span> before they reach users.</li> <li><strong>Disadvantage:</strong> If any of the installed provider modules has an unrelated issue, AI module CI will go red and block merges until it is resolved upstream.</li> <li><strong>Disadvantage:</strong> Slightly slower CI and more moving parts to maintain.</li> </ul> <h3 id="summary-ai-usage">AI usage (if applicable)</h3> <p>[x] AI Assisted Issue<br> This issue was generated with AI assistance, but was reviewed and refined by the creator.</p> <p>[ ] AI Assisted Code<br> [ ] AI Generated Code<br> [ ] Vibe Coded</p> <p>- <strong>This issue was created with the help of AI</strong></p>