Resolve "Canvas AI: Add a dev page builder agent and wire the placement tools"

Ahmad Khalilrequested to merge
issue/canvas-3591921:3591921-canvas-ai-add-a-dev-page-builder-agent-and-wire-the-placement-tools into 1.x

For testing steps, please refer: #3591921 (comment 1940926)

What this MR does

Adds the dev page builder agent (canvas_dev_page_builder_agent) and wires the new placement tools end-to-end through canvas_dev_ai's client-side hop loop. One agent now covers what the four live agents (canvas_page_builder_agent, canvas_template_builder_agent, canvas_title_generation_agent, canvas_metadata_generation_agent) split between them: building pages, extending them, editing components already on the page, and setting page title/description. The four live agents are untouched — consolidation and the later rename to canvas_page_builder_agent stay out of scope per the issue.

The agent

  • New config entity modules/canvas_dev_ai/config/install/ai_agents.ai_agent.canvas_dev_page_builder_agent.yml, shipped by canvas_dev_ai (dev-only lifecycle: installed with the dev module, removed on its uninstall), based on the POC prompt with the POC-only artifacts removed (rag_tool, the [canvas_ai:execution_strategy] token).
  • Tools wired: canvas_ai:place_components, canvas_ai:edit_components, canvas_ai:get_component_details, canvas_ai:set_page_value. place_components has Restrict multiple calls enabled with the message from the issue. max_loops: 50.
  • Information tools: canvas_ai:get_component_context with catalog_only: true on the first hop only (available_on_loop: [1]), canvas_ai:get_current_layout on every hop.
  • canvas_ai:get_component_context gained an optional boolean catalog_only context: when set, it returns the lean id/name/description catalog from CanvasAiComponentContextHelper::getComponentCatalog(). Absent/false keeps the full output, so the live agents are unaffected.
  • Prompt behavior per the follow-up comment on the issue:
    • Plan gate anchored to the agreed rule: placing or editing more than one component, or a distinct section, requires describing the plan and asking first; a single unambiguous action acts directly. A "Gate replies" rule classifies the user's answer (approval / adjustment / unrelated) — an adjustment produces the full revised plan and stops at the gate again, in a text-only turn.
    • In-progress narration: placements/edits are narrated present-continuous ("Adding the hero row now — it'll appear shortly."); past tense only once the work is done.
    • No execution-strategy toggle: row-by-row is the only behavior; the prompt's "Build sequence" section states there is no batch mode, and nothing configurable was added.

Backend wiring

  • CanvasDevAiBuilder now instantiates the agent from canvas_dev_ai.settings:main_agent (previously hardcoded to canvas_component_agent), with a fallback for pre-existing settings. The shipped default cannot name the new agent directly — core installs simple config before config entities, so the ConfigExists constraint would fail mid-install; instead canvas_dev_ai_install() flips main_agent after config install, and canvas_dev_ai_post_update_0001 covers existing installs (creates the agent preserving uuid/_core, flips main_agent only if it still has the old shipped default).
  • The new agent is selectable in the Agents & Tools form.
  • PlaceComponents/EditComponents now catch YAML ParseException and return an instructive error (quote string values; unquoted date-like values such as 2233-33-33 are parsed as invalid dates). Raw parse errors previously gave the model nothing to act on and caused identical-retry loops; instructive errors make it self-correct, matching how it already recovers from validation errors.

Frontend (dev wizard only — AiWizard.tsx untouched)

Three bugs found during live testing, all in how hop responses were applied:

  1. Race: receiveMessage() fired the operations handler via setTimeout(0) and dropped the promise, so the next hop's current_layout was built before placements were applied. Handlers are extracted to messageHandlers.ts (processAiMessage()), and every handler — operations included — is awaited before the hop resolves.
  2. UUID chaining: the operations handler generated a fresh client-side UUID for each placed component, while place_components' tool result tells the model to chain reference_uuid on the backend-assigned UUID — every follow-up placement failed with "Component with UUID … not found in layout". The handler now passes predefinedUUID through addNewComponentToLayout.
  3. Edits were dropped: edit_components' structured output (component_updates) had no frontend handler — edits validated server-side and reported success, but the canvas never changed. A new handler applies them via the existing layoutUtils.updateExistingComponentValues.

Requirements

composer.json require-dev: drupal/ai ^1.4, drupal/ai_agents ^1.3 (needed for available_on_loop and per-tool restrict_multiple_calls); README version text updated.

Test coverage

  • CanvasDevPageBuilderAgentEndToEndTest — kernel E2E driving three real HTTP hops through /admin/api/canvas/ai-dev against recorded echoai fixtures: hop 1 parks the hero placement, hop 2 executes it and chains the heading below the hero's backend UUID from the tool result, hop 3 sends the layout containing the hero and closes the turn. UUIDs deterministic via a stubbed uuid service.
  • CanvasDevPageBuilderAgentConfigTest — agent created via real module install (exercises hook_install), tool registration, explicit config-schema validation, restrict-multiple-calls settings, information-tool hop config, post-update create/flip/preserve semantics.
  • Extended: PlaceComponentsTest/EditComponentsTest (instructive YAML-parse errors), GetComponentContextTest (catalog_only on/off), CanvasDevAiAgentSelectionFormTest.
  • messageHandlers.test.tsx — vitest: processAiMessage() resolves only after the operations handler finishes (fake timers, dispatch ordering, navigate-before-resolve), backend-UUID preservation, component_updates application, page-data application, error handling.

Testing instructions

Setup

  • Check out the branch, then build the UI and clear caches — mandatory, the branch ships frontend fixes:

    cd <canvas>; npm ci && npm run --workspace=@drupal-canvas/ui build
drush cr

  • Fresh install: drush pm:enable canvas canvas_ai canvas_dev_ai. Pre-existing canvas_dev_ai install: run drush updb instead and confirm canvas_dev_ai_post_update_0001_add_dev_page_builder_agent appears in the log.

  • Configure an AI provider with function calling and set the default Chat model to a current-generation model (gpt-5.2 / Claude Sonnet class). Older models (verified with gpt-4o) do not follow the plan-gate instructions reliably — that is a model limitation, not this branch.

  • Log in with use Drupal Canvas AI.

Config checks

  • /admin/config/ai/tools-automation/agents lists Drupal Canvas Dev Page Builder Agent; its form shows exactly the four new tools, "Restrict multiple calls" on place_components with the message from the issue, max loops 50.
  • On the Canvas AI settings Agents & Tools tab, Main agent is preselected to the dev page builder agent; selecting it as a Tool too is rejected.

Frontend (dev wizard only — AiWizard.tsx untouched)

Three bugs found during live testing, all in how hop responses were applied:

  1. Race: receiveMessage() fired the operations handler via setTimeout(0) and dropped the promise, so the next hop's current_layout was built before placements were applied. Handlers are extracted to messageHandlers.ts (processAiMessage()), and every handler — operations included — is awaited before the hop resolves.
  • Single edit applies in place: "Change that heading to say 'Hello there' and make it an H2" → the heading visibly changes on the canvas (no duplicate), and its props form shows the new values.
  • Bulk edit gates, then applies: "Make all the headings on this page title case" → plan + approval, then every listed heading changes.
  • Metadata gate: after a build on a page with an empty title/description, the agent proposes both, stops for approval, and the page's Title/Description fields update after "yes".

Network verification (DevTools → Network, filter /admin/api/canvas)

  • Every turn POSTs to /admin/api/canvas/ai-dev (never /admin/api/canvas/ai), with an X-CSRF-Token header; the build turn is a burst sharing one request_id, should_continue: true until the final hop, progress growing per hop.
  • Placement hops carry operations (components with id/uuid/nodePath/fieldValues); edit turns carry component_updates.
  • The race/UUID fixes: take a hop whose response contains operations, note a component's uuid, and confirm the next request's current_layout.regions.content.components contains that same uuid at the matching index.

Automated

composer run phpunit -- modules/canvas_ai/tests/src/Kernel/
composer run phpunit -- modules/canvas_ai/tests/src/Functional/Form/CanvasDevAiAgentSelectionFormTest.php
npm run --workspace=@drupal-canvas/ui test -- src/components/aiExtension/

(Latest local run: 161 kernel tests / 2218 assertions, 3 functional, 8 vitest — all passing; phpcs/phpstan/eslint/prettier/cspell clean on all changed files.)

Known limitations and follow-ups

  • Mid-build stall (rare): a model can end a completion without a tool call mid-build; the client loop has no automatic re-prompt, so the turn ends. Recovery is replying "continue" (the agent resumes from the current layout, no data loss). The prompt hardens against this; the mechanical fix (re-prompting on unfinished work)

Frontend (dev wizard only — AiWizard.tsx untouched)

Three bugs found during live testing, all in how hop responses were applied:

  1. Race: receiveMessage() fired the operations handler via setTimeout(0) and dropped the promise, so the next hop's current_layout was built before placements were applied. Handlers are extracted to messageHandlers.ts (processAiMessage()), and every handler — operations included — is awaited before the hop resolves.
  2. UUID chaining: the operations handler generated a fresh client-side UUID for each placed component, while place_components' tool result tells the model to chain reference_uuid on the backend-assigned UUID — every follow-up placement failed with "Component with UUID … not found in layout". The handler now passes predefinedUUID through addNewComponentToLayout.
  3. Edits were dropped: edit_components' structured output (component_updates) had no frontend handler — edits validated server-side and reported success, but the canvas never changed. A new handler applies them via the existing layoutUtils.updateExistingComponentValues.

Requirements

composer.json require-dev: drupal/ai ^1.4, drupal/ai_agents ^1.3 (needed for available_on_loop and per-tool restrict_multiple_calls); README version text updated.

Test coverage

  • CanvasDevPageBuilderAgentEndToEndTest — kernel E2E driving three real HTTP hops through /admin/api/canvas/ai-dev against recorded echoai fixtures: hop 1 parks the hero placement, hop 2 executes it and chains the heading below the hero's backend UUID from the tool result, hop 3 sends the layout containing the hero and closes the turn. UUIDs deterministic via a stubbed uuid service.
  • CanvasDevPageBuilderAgentConfigTest — agent created via real module install (exercises hook_install), tool registration, explicit config-schema validation, restrict-multiple-calls settings, information-tool hop config, post-update create/flip/preserve semantics.
  • Extended: PlaceComponentsTest/EditComponentsTest (instructive YAML-parse errors), GetComponentContextTest (catalog_only on/off), CanvasDevAiAgentSelectionFormTest.
  • messageHandlers.test.tsx — vitest: processAiMessage() resolves only after the operations handler finishes (fake timers, dispatch ordering, navigate-before-resolve), backend-UUID preservation, component_updates application, page-data application, error handling.

Testing instructions

Setup

  • Check out the branch, then build the UI and clear caches — mandatory, the branch ships frontend fixes:

    cd <canvas>; npm ci && npm run --workspace=@drupal-canvas/ui build
drush cr

  • Fresh install: drush pm:enable canvas canvas_ai canvas_dev_ai. Pre-existing canvas_dev_ai install: run drush updb instead and confirm canvas_dev_ai_post_update_0001_add_dev_page_builder_agent appears in the log.

  • Configure an AI provider with function calling and set the default Chat model to a current-generation model (gpt-5.2 / Claude Sonnet class). Older models (verified with gpt-4o) do not follow the plan-gate instructions reliably — that is a model limitation, not this branch.

  • Log in with use Drupal Canvas AI.

Config checks

  • /admin/config/ai/tools-automation/agents lists Drupal Canvas Dev Page Builder Agent; its form shows exactly the four new tools, "Restrict multiple calls" on place_components with the message from the issue, max loops 50.
  • On the Canvas AI settings Agents & Tools tab, Main agent is preselected to the dev page builder agent; selecting it as a Tool too is rejected.
  • drush config:get canvas_dev_ai.settings main_agent → canvas_dev_page_builder_agent (a deliberately different choice must be preserved across drush updb).

Chat scenarios (clean Canvas page, AI panel in the editor)

  • Plan gate: "Build me a landing page for a coffee subscription service" → plain-text plan + approval question, nothing placed.
  • Adjustment loops back: "swap the testimonials for a FAQ section" → complete revised plan + asks again, text-only turn, still nothing placed.
  • Approve: "proceed" → the whole revised plan builds, sections appearing one at a time top-to-bottom (~1 s apart), narration present-continuous during the build, past tense only in the closing line. Occasional single-shot corrective retries ("wrong slot name — correcting…") are designed behavior.
  • Section-sized request gates: "Add a pricing section" → brief plan first.
  • Single action doesn't gate: "Add a heading that says 'Welcome' at the bottom of the page" → places directly.
  • Single edit applies in place: "Change that heading to say 'Hello there' and make it an H2" → the heading visibly changes on the canvas (no duplicate), and its props form shows the new values.
  • Bulk edit gates, then applies: "Make all the headings on this page title case" → plan + approval, then every listed heading changes.
  • Metadata gate: after a build on a page with an empty title/description, the agent proposes both, stops for approval, and the page's Title/Description fields update after "yes".

Network verification (DevTools → Network, filter /admin/api/canvas)

  • Every turn POSTs to /admin/api/canvas/ai-dev (never /admin/api/canvas/ai), with an X-CSRF-Token header; the build turn is a burst sharing one request_id, should_continue: true until the final hop, progress growing per hop.
  • Placement hops carry operations (components with id/uuid/nodePath/fieldValues); edit turns carry component_updates.
  • The race/UUID fixes: take a hop whose response contains operations, note a component's uuid, and confirm the next request's current_layout.regions.content.components contains that same uuid at the matching index.

Automated

composer run phpunit -- modules/canvas_ai/tests/src/Kernel/
composer run phpunit -- modules/canvas_ai/tests/src/Functional/Form/CanvasDevAiAgentSelectionFormTest.php
npm run --workspace=@drupal-canvas/ui test -- src/components/aiExtension/

(Latest local run: 161 kernel tests / 2218 assertions, 3 functional, 8 vitest — all passing; phpcs/phpstan/eslint/prettier/cspell clean on all changed files.)

Known limitations and follow-ups

  • Mid-build stall (rare): a model can end a completion without a tool call mid-build; the client loop has no automatic re-prompt, so the turn ends. Recovery is replying "continue" (the agent resumes from the current layout, no data loss). The prompt hardens against this; the mechanical fix (re-prompting on unfinished work) belongs to the hop-loop work in #3591777.
  • AiWizard.tsx dedup: the live wizard still carries its own copy of the message handlers; switching it to the shared messageHandlers.ts is a follow-up so this MR does not change production-wizard timing.
  • Live-LLM eval group: the canvas_ai_agents_test group yaml still asserts the old set_component_structure tool; updating it is deferred to the consolidation issue.
  • Testing on Byte-theme sites: card-pricing/hero-blog components error with [byte_theme:heading/url] Invalid URL format … "No URL" wherever SDC validation runs — an upstream theme bug, filed with a fix at byte_theme#3591007. Not caused by this branch; ignore during review.
  • Observation for a possible canvas_ai follow-up: placement-time validation does not assert SDC prop format constraints (render-time does), and the enum-violation message doesn't distinguish 48 from "48" — both occasionally cost the model an extra retry.
Edited by AKHIL BABU

Merge request reports