3592021: Add AiResponseValidator::formatErrors() to render accumulated tool...

Ahmad Khalilrequested to merge
issue/canvas-3592021:3592021-canvas-ai-unify-error-handling-in-canvas-component-agents-tools into 1.x

Applies the error-handling pattern from #3591958 to the tools canvas_component_agent uses. The two tools that validate agent input, ai_agent:create_component and ai_agent:edit_component_js, now collect every error and report them together through AiResponseValidator::formatErrors(), so the model can fix all of them in one retry instead of one per turn. Nothing is applied while any error exists.

Tools in scope

Tool Status
ai_agent:create_component Changed — see below
ai_agent:edit_component_js Changed — see below
ai_agent:get_js_component No change — two sequential single checks (component missing / no view access); nothing to accumulate
ai_agent:get_node_fields No change — access check already throws first; its only other check is a single early return (unknown node type)
ai_agent:get_props_type No change — no validation at all

What changed and why

AiResponseValidator::formatErrors() (commit 1)

Added the formatting helper from #3591958 / !1584 with identical code, because that MR is not merged yet and this branch needs it. Whichever MR lands second drops its copy on rebase.

AiGeneratedJsComponentPropsAndSlotsTrait::transformSlotsMetadata() (commit 2)

Before: threw on the first invalid slot, so a call with three bad slots needed three model turns.

After: takes an array &$errors accumulator (keyed Slots for non-array JSON, Slot <index> per invalid slot), skips the invalid slot and keeps going. The camelCase check is skipped for a slot that has no id or name, since the value it would compare is invalid. Message strings are unchanged.

CreateComponent / EditComponentJs (commit 2)

Before: three different failure styles in one method — throw (duplicate name / unknown component, slot errors, ConstraintViolationException), an early return with setOutput() (@apply in the CSS), and a catch-all that wrapped the first error in Yaml::dump(['error' => …]).

After:

  • Every existing check appends to $errors['Component'], $errors['Slots'] or $errors['Slot N'] instead of throwing or returning early.
  • Entity validation (JavaScriptComponent::createFromClientSide()->getTypedData()->validate()) still runs alongside a name collision or an @apply error, since neither affects the structure being validated, so those violations are reported in the same turn. It is skipped only when its input is invalid: any slot error, or (for edit) an unknown component whose name it would read.
  • On any error: formatErrors() → logger->warning() (channel canvas_ai) → setOutput() → return. setStructuredOutput() is not called, so nothing reaches the client.
  • The generic catch (\Exception $e) stays as a last resort for unexpected failures and now outputs a plain Failed to process Javascript component data: … string.
  • The Yaml::dump(['error' => …]) wrapper and the "CanvasBuilder::render() also YAML parsable output" comments were removed: CanvasBuilder has only read getStructuredOutput() since c78a54e9 (#3555464 (closed)); readable output goes to the model only.
  • AiResponseValidator is injected via canvas_ai.response_validator, as in PlaceComponents / EditComponents.

Example output for a create call with an existing name, @apply in the CSS and two invalid slots:

`Nothing was applied. Fix every error listed below and call the tool again.

Component

  • The component with same name already exists.
  • @apply directives are not supported.

Slot 0

  • Each slot must include both an "id" and a "name". Slot "No Id" is missing one of them.

Slot 1

  • The slot "id" must be the camelCase of the slot "name". Got id "cta_content" for name "CTA Content"; expected "ctaContent". `

Tests (commit 2)

CreateComponentTest and EditComponentJsTest: the error tests are now one data-provider test each (testErrorsAreReportedTogether, 11 and 8 cases) asserting the exact message and [] === getStructuredOutput(). New cases cover: several errors across sections in one call, every invalid slot reported, unknown component plus invalid slot, and validation still running when the name already exists. Success tests are untouched. The private assertYamlError() helpers are gone with the YAML wrapper.

Out of scope (follow-ups)

  • create_component, edit_component_js and get_js_component have no use Drupal Canvas AI permission check; only the /admin/api/canvas/ai route enforces it. The issue keeps access checks "unchanged", so none were added.
  • props_metadata that is not a JSON array, and prop entries missing id/name/type/example, are still silently dropped. Turning those into errors is a new validation, not a conversion of an existing one.

Testing instructions

Automated

  • [ ] composer run phpunit -- modules/canvas_ai/tests/src/Kernel/Plugin/AiFunctionCall/CreateComponentTest.php passes (14 tests)
  • [ ] composer run phpunit -- modules/canvas_ai/tests/src/Kernel/Plugin/AiFunctionCall/EditComponentJsTest.php passes (11 tests)
  • [ ] Regression: GetJsComponentTest, GetNodeFieldsTest, PlaceComponentsTest, EditComponentsTest, SetAIGeneratedComponentStructureTest, SetAIGeneratedTemplateDataTest, Kernel/Agents/CanvasComponentAgentEndToEndTest pass (63 tests locally)
  • [ ] composer run lint (phpcs + phpstan) and npx cspell report nothing for the changed files
  • [ ] Pipeline is green

Manual, without a model (drush)

  • [ ] Run a drush php:script that instantiates ai_agent:create_component via plugin.manager.ai.function_calls with component_name set to an existing code component's machine name, css_structure containing @apply, and slots_metadata with two invalid slots. getReadableOutput() is the message shown above and getStructuredOutput() is []
  • [ ] Same for ai_agent:edit_component_js with an unknown component_machine_name and an invalid slot: a ## Component and a ## Slot 0 section, [] structured output
  • [ ] A valid create and a valid edit still return the success message and the same structured output as before
  • [ ] Reports → Recent log messages, type canvas_ai: one warning per failed call containing the full message

Manual, in the UI (needs a configured chat provider)

Setup: a page with Canvas, a user with use Drupal Canvas AI, and an existing code component (for example one named "Hero Banner").

  • [ ] Errors reach the model as one list. In the Canvas editor open the AI panel and ask: "Create a code component named Hero Banner with a slot whose id is main_content and name Main Content, and use @apply font-bold in its CSS." Expected: the agent does not create the component on the first tool call; it either reports back that the name already exists, @apply is not supported and the slot id must be mainContent — all together, in one reply — or it corrects all three and creates a differently named component in its next tool call. Reports → Recent log messages (type canvas_ai) shows a single warning with the Nothing was applied… message listing all three errors under ## Component and ## Slot 0
  • [ ] Edit path. Select the existing "Hero Banner" component and ask: "Add a slot with id cta_content and name CTA Content." Expected: same behavior — one warning with a ## Slot 0 section (expected "ctaContent"); the agent retries with ctaContent or reports the error. No partial change to the component
  • [ ] Happy path unchanged. Ask: "Create a Card component with a heading prop and a children slot." Expected: the component is created, appears in the code components library with the prop and the slot, and the chat reply is the usual one-sentence confirmation. Then, with it selected, ask: "Add a subtitle prop." Expected: the edit is applied and the reply mentions the update (and lists any props/slots left out of the update, as before)
  • [ ] No error is surfaced as a YAML error: block anywhere in the chat

Closes #3592021

AI-Generated: Yes (Used Claude Code to implement the changes, the kernel test cases and to run the local test and lint tooling; every change was reviewed and verified by the human author).

Merge request reports