#3591952 Canvas AI: Prevent the selected Tool from changing during a chat turn

Ahmad Khalilrequested to merge
issue/canvas-3591952:3591952-canvas-ai-prevent-the-selected-tool-from-changing-during-a-chat-turn into 1.x

Summary

One dev chat turn runs as several POSTs to /admin/api/canvas/ai-dev under one request_id, and each of them carries the active Tool as selected_tool. Two gaps let the Tool change in the middle of a turn:

  • The Tools menu and the active Tool pill stayed interactive while a turn was running, and the chat read the current selection every time it built a hop's body.
  • The agent state parked between hops held no agent ID, so the controller could not tell that a later hop resolved a different agent than the one whose chat history it was about to restore.

A turn could therefore start with one agent and be resumed by another, with undefined results. This MR fixes the Tool for the whole turn on both sides:

  • Backend: the parked state records the agent that parked it, and a continuation hop that resolves a different agent is rejected with an error response.
  • Frontend: the Tool is locked while a turn runs and released when the turn ends, including when it ends in an error or is aborted.

Closes #3591952 (closed). Part of #3591777.

What changed

Backend

  • modules/canvas_ai/src/CanvasAiTempStore.php
    • setStoredAgentState(string $job_id, string $agent_id, array $state) stores ['agent_id' => ..., 'state' => ...] under the existing agent_state_<request_id> key, so set, get and delete stay atomic and the ai_agents toArray() contract is untouched.
    • getStoredAgentState() returns that record, typed array{agent_id: string, state: array}|null. A malformed or legacy record reads as "not paused" and seeds a fresh agent.
  • modules/canvas_dev_ai/src/Controller/CanvasDevAiBuilder.php
    • render() reads the record once, and after resolveAgentId() compares the stored agent ID with the resolved one. On a mismatch it deletes the parked state (the turn is over, same as the determineSolvability() failure path) and returns the standard error response: status: false, should_continue: false, message The selected tool cannot change during a turn.
    • The comparison uses the resolved agent ID rather than the raw selected_tool: that is the real invariant (fromArray() must be fed to the agent that wrote the state), and it also catches a Tool cleared mid-turn, which resolves the main agent.
    • prepareAgent() receives the already-read state instead of re-reading the tempstore. The @todo pointing at this issue is removed.

Frontend (ui/src/components/aiExtension)

  • AiWizardDev.tsx
    • New isTurnInProgress state, set when a turn starts and cleared in a new finally. The finally also runs on the AbortError early return in the catch (panel unmount), which previously skipped the trailing code.
    • The Tool is snapshotted once per turn and used by every hop's buildContext(), mirroring the existing activeComponentUuid snapshot, so all hops of a turn carry the same value regardless of UI state.
    • disabled={isTurnInProgress} is passed to the pill and the selector. No MemoDeepChat prop changes identity, so the typed prompt survives the state flips.
  • AiToolSelector.tsx / .module.css
    • New disabled prop. Rows are natively disabled (muted, no hover highlight, selected row keeps its tint) and a note reads Tools cannot be changed while a turn is in progress.
    • The deep-chat trigger still opens the menu on purpose. deep-chat's setState('disabled') on a custom button is visual only (its click handler fires regardless), and a menu of disabled rows plus a note tells the user why nothing is selectable without coupling to deep-chat's state mutation.
  • ActiveToolPill.tsx / .module.css
    • New disabled prop: the remove button is natively disabled and the pill drops its hover highlight. The tooltip stays "Remove" because Radix tooltips do not open on disabled buttons; the explanation lives in the selector note.
    • data-testid="canvas-ai-active-tool" for browser tests.

Tests

  • Kernel (modules/canvas_ai/tests/src/Kernel)
    • Agents/CanvasDevAiAgentRoutingTest: testToolCannotChangeDuringTurn() (data provider: another Tool, no Tool) asserts the hop is rejected before createInstance(), with the exact message, and that the parked state is dropped; testSameToolContinuesTheTurn() asserts the same Tool resumes the turn and the parked state is restored through fromArray().
    • Agents/CanvasComponentAgentEndToEndTest: testToolCannotChangeDuringTurn() drives a real echoai hop that parks state under the component agent, then a tool-less hop 2 that is rejected. The red button component setup is shared through createRedButtonComponent().
    • CanvasAiDevControllerTest: the tempstore seed uses the new signature, parked under the main agent the tool-less hop resolves.
  • Vitest: AiToolSelector.test.tsx and ActiveToolPill.test.tsx cover the enabled and disabled behavior.
  • Playwright: tests/src/Playwright/tests/isolatedPerTest/aiDev.spec.ts, new test "The Tool is fixed for the turn". It holds the turn's second request until the locked UI has been asserted, checks both requests carried the same selected_tool, then changes and clears the Tool after the answer.

Testing instructions

Automated

  • [ ] composer run phpunit -- modules/canvas_ai/tests/src/Kernel/Agents/CanvasDevAiAgentRoutingTest.php
  • [ ] composer run phpunit -- modules/canvas_ai/tests/src/Kernel/CanvasAiDevControllerTest.php
  • [ ] composer run phpunit -- modules/canvas_ai/tests/src/Kernel/Agents/CanvasComponentAgentEndToEndTest.php
  • [ ] composer run phpunit -- modules/canvas_ai/tests/src/Kernel/Agents/CanvasDevPageBuilderAgentEndToEndTest.php (the page builder path round-trips the new record shape)
  • [ ] npm run --workspace=@drupal-canvas/ui test -- src/components/aiExtension
  • [ ] npm run test:playwright -- tests/src/Playwright/tests/isolatedPerTest/aiDev.spec.ts
  • [ ] composer run lint and npm run lint

Manual, in the UI

Setup:

  • [ ] Rebuild the UI: npm install && npm run build in the module root (ui/dist is not committed).
  • [ ] drush pm:enable -y canvas_dev_ai && drush cr
  • [ ] A default chat provider is configured at /admin/config/ai/settings (the steps below make real provider calls; each turn is two short requests).
  • [ ] At /admin/config/ai/canvas-ai-agent-selection: make sure Main agent is not "Drupal Canvas Component Agent" (pick "Drupal Canvas Page Builder Agent" if it is), tick Drupal Canvas Component Agent under Tools, save, then drush cr.
  • [ ] Open a Canvas page in the editor, for example /canvas/editor/canvas_page/1, and open DevTools with the Network tab filtered on ai-dev.

Test 1: the Tool is locked while a turn runs and released when it ends

  • [ ] Click the sparkle icon in the top bar (Open AI Panel). The "Drupal Canvas AI" chat opens.
  • [ ] Click the blue sliders icon at the bottom-left of the chat input (next to "+"). A popover titled AVAILABLE TOOLS lists "Drupal Canvas Component Agent". (If the icon is missing, the Tool was not saved or caches are stale.)
  • [ ] Click Drupal Canvas Component Agent. The popover closes and a blue pill "Drupal Canvas Component Agent" with an × appears above the chat.
  • [ ] Type Create a red button code component and press Enter.
  • [ ] While the three-dot loader is visible, before the answer appears:
    • [ ] Hover the pill: it does not turn dark blue and the cursor over the × is "not allowed". Click the ×: the pill stays.
    • [ ] Click the sliders icon: the popover opens with the note Tools cannot be changed while a turn is in progress. and the row is greyed out. Click the row: nothing changes. Press Escape.
    • [ ] If the turn finishes too quickly, throttle the connection in the Network tab (for example "Slow 3G") and send the message again.
  • [ ] In the Network tab there are exactly two POSTs to /admin/api/canvas/ai-dev. Both payloads carry the same request_id and "selected_tool": "canvas_component_agent". The first response has "should_continue": true, the second "should_continue": false plus a message.
  • [ ] After the answer appears: hovering the pill highlights it again, the popover shows no note and an enabled row, clicking the × removes the pill, and selecting the Tool again brings it back.

Test 2: a turn that ends in an error also releases the lock

  • [ ] In the Network tab, right-click an earlier ai-dev request and choose Block request URL.
  • [ ] With the Tool selected, send Hello.
  • [ ] An error message appears in the chat, and right after it the pill's × and the Tools rows are usable again.
  • [ ] Unblock the URL again (Network request blocking panel).

Test 3: the server rejects a continuation that changes the Tool (replayed from the Console, since the UI can no longer send it)

  • [ ] With the Tool selected, send Create a blue button code component and wait for the answer.
  • [ ] Right-click the first of that turn's two ai-dev requests, then Copy → Copy as fetch.
  • [ ] In the Console, paste it, change the request_id value in the body to a new unique string (for example manual-turn-1), keep "selected_tool":"canvas_component_agent", and run it. The new request's response shows "status": true and "should_continue": true (this parks the agent; one provider call).
  • [ ] Paste the same fetch again with request_id still manual-turn-1, delete "selected_tool":"canvas_component_agent" (and the comma before it) from the body, and run it.
  • [ ] The response is {"status": false, "message": "The selected tool cannot change during a turn.", "should_continue": false, "progress": "..."}. No provider call is made for this hop.
  • [ ] The "same Tool continues normally" case is proven by Test 1: the second hop carried the same Tool and was processed.

Cleanup:

  • [ ] Delete the created components (Red Button, and Blue Button if it was created): from the code component library, or drush php:eval '$s=\Drupal::entityTypeManager()->getStorage("js_component"); foreach ($s->loadMultiple(["red_button","blue_button"]) as $c) { $c->delete(); }'
  • [ ] Untick the Tool at /admin/config/ai/canvas-ai-agent-selection if it was not enabled before.

AI-Assisted: Yes (Claude Code). The review methodology, test runs, and verification steps described above were executed in a real local environment and the results independently confirmed by me.

Edited by Ahmad Khalil

Merge request reports