Canvas AI: Refine the orchestrator prompt
>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3554026. --> Reported by: [akhil babu](https://www.drupal.org/user/3632866) Related to !289 !261 >>> <h3 id="overview">Overview</h3> <h3>Proposing 3 Improvements for the Orchestrator</h3> <h3>1. Orchestrator should handle the <code>entity_type</code> check</h3> <p>Out of all five sub-agents of the orchestrator, the following work only with <strong>Canvas page entities</strong>:</p> <ul> <li>Drupal Canvas Metadata Generation Agent</li> <li>Drupal Canvas Title Generation Agent</li> <li>Drupal Canvas Page Builder Agent</li> <li>Drupal Canvas Template Builder Agent</li> </ul> <p>The remaining agent, <strong>Drupal Canvas Component Agent</strong>, doesn&rsquo;t depend on the entity type.<br> Currently, the entity type check happens inside the individual sub-agents. We should consider moving this check to the orchestrator level so that </p> <ol> <li> One API request can be saved since non-relevant sub-agents won&rsquo;t be invoked, thus reducing cost.<br> <br><em>(For example, if the request is for a page-building task, invoking the Page Builder Agent passes the entire component context along with the system prompt, which is costly.)</em> </li> <li>Sub-agents won&rsquo;t need to check the entity type, allowing their prompts to focus purely on their specific tasks.</li> <li>The user would get faster responses.</li> </ol> <h3>2. Orchestrator should generate title and metadata for page-building and template-generation requests</h3> <p>Currently, the Template Builder and Page Builder agents invoke the Title and Metadata Generation agents as tools when needed.<br> This responsibility can instead be handled directly at the orchestrator level.</p> <p>These two changes will make both page building and template generation more reliable.<br> At the moment, both agents have complex instructions to generate outputs in specific formats. Adding extra instructions to check the entity type and generate title/metadata increases the likelihood of hallucinated or inconsistent output.</p> <h3>3. Orchestrator should accurately detect the sub-agent for each task</h3> <p>In many cases, the orchestrator incorrectly delegates template and page-building tasks to the Component Agent.<br> See this comment: <a href="https://www.drupal.org/project/canvas/issues/3533079#comment-16317495">https://www.drupal.org/project/canvas/issues/3533079#comment-16317495</a></p> <p>We should update the orchestrator&rsquo;s prompt logic so it can invoke the ppropriate sub-agent more reliably.</p> <h3 id="proposed-resolution">Proposed resolution</h3> <ul> <li>Create a new token that provides detailed context around the current page or code component to the orchestratior</li> <li>Update the prompt of the orchestrator agent. Use this token. Move the title/metadata generation parts and the entity type validation parts to it</li> <li>Update the prompt of the page builder agent. Remove the title/metadata generation parts and the entity type validation parts.</li> </ul> <h3 id="ui-changes">User interface changes</h3>
issue