>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3584852. --> Reported by: [marcus_johansson](https://www.drupal.org/user/385947) Related to !1501 >>> <p>[Tracker]<br> <strong>Update Summary: </strong>[One-line status update for stakeholders]<br> <strong>Short Description: </strong>The patch-test system prompt step relies on a haiku response which is ambiguous to verify and unclear to non-native readers - switch it to the deterministic "banana expert" prompt instead.<br> <strong>Check-in Date: </strong>MM/DD/YYYY<br> [/Tracker]</p> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>While doing QA testing against the <a href="https://project.pages.drupalcode.org/ai/1.3.x/contribute/testing/full_patch_test/">full patch test documentation</a>, the "Test the base functionality" step sends the tester into <code>docs/developers/testing_an_ai_provider.md</code>, which in turn uses a haiku as the system-prompt verification test:</p> <p><code>You are a helpful assistant that always responds in a haiku.</code></p> <p>This is a poor test for a few reasons. It assumes the tester knows what a haiku is (a 5-7-5 syllable Japanese poem) - which is not universally obvious, especially to non-native English readers contributing to the module. Even for testers who do know the form, verifying that a response is actually a haiku requires counting syllables, which is subjective and model-dependent. The test is effectively ambivalent - a loose rhymed couplet could plausibly "pass" and a model that doesn't really follow the system prompt could still produce something that looks vaguely haiku-shaped.</p> <p>A better alternative is the "banana test": use a system prompt that says <code>Always mention banana at least once</code>. That instruction is unambiguous to verify - either the word "banana" appears in the response or it doesn't. It works regardless of language, does not require any cultural or literary knowledge, and cannot be accidentally "passed" by a model that ignores the system prompt.</p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <ul> <li>Rewrite the "System Prompt Test" section of <code>docs/developers/testing_an_ai_provider.md</code> to use the banana test instead of the haiku test.</li> <li>System prompt: <code>Always mention banana at least once</code>.</li> <li>Use a neutral user message (for example <code>Tell me about the weather today.</code>) and describe the expected behavior as: the response must contain the word "banana" at least once.</li> <li>Remove the link to the Wikipedia haiku article since it is no longer needed.</li> <li>Double-check any other place in the patch test flow that relies on the haiku prompt and update it for consistency.</li> </ul> <h3 id="summary-ai-usage">AI usage (if applicable)</h3> <p>[x] AI Assisted Issue<br> This issue was generated with AI assistance, but was reviewed and refined by the creator.</p> <p>[ ] AI Assisted Code<br> [ ] AI Generated Code<br> [ ] Vibe Coded</p> <p>- <strong>This issue was created with the help of AI</strong></p>