feat: #3586051 Add a per-tool setting to restrict a tool from being called...

Ahmad Khalilrequested to merge
issue/ai_agents-3586051:3586051-add-a-per-tool-setting-to-restrict-a-tool-from-being-called-more-than-once-per-response into 1.4.x

feat: #3586051 Add a per-tool setting to restrict a tool from being called more than once per response

What this MR does

Implements the per-tool "Restrict to one call per response" setting proposed in this issue.

Why: Some tools must run at most once per LLM response because each call depends on the previous call's result. The motivating case is Canvas's client-side orchestration, where place_components places one component per step and later placements reference the UUID returned by the previous call — two calls in one response can never be valid. Other tools (e.g. edit_components) may legitimately be called several times, so this must be opt-in per tool.

Changes

src/PluginBase/AiAgentEntityWrapper.php (runtime behavior)

  • Two new tool_settings keys per tool: restrict_multiple_calls (bool, default FALSE) and multiple_call_error_message (string, default '' = use the default message). New accessors toolRestrictsMultipleCalls() / getMultipleCallErrorMessage() follow the existing pattern used by return_directly / use_artifacts, including the functionsOverride fallback.
  • In determineSolvability(), tool calls returned by the LLM are now all converted to function-call objects before anything is queued, then counted per plugin ID. If a restricted tool appears more than once, all tool calls in that response are rejected: none execute, an error is logged to the ai_agents channel, and the agent retries (bounded by the existing max_loops guard).
  • Rejections are written as one tool-role message per tool call ID — not a system message — because the assistant message containing the tool_calls is already in chat history, and OpenAI-style APIs require a tool result for every call ID. Restricted calls get the configured/default error (with [tool_name] substituted); other calls in the same response get a "not executed because tool X may only be called once" message.

src/Form/AiAgentForm.php

New checkbox plus a "Multiple call error message" textarea shown via #states only when the checkbox is on (mirrors the description_override pattern). The message is cleared on save when the restriction is off.

src/Plugin/ModelerApiModelOwner/Agent.php

Parity defaults and form fields for the modeler UI. Its save path copies settings keys generically, so no other changes were needed there.

Backward compatibility: all reads default via ?? FALSE / ?? '', so existing agents need no update hook, and the tool_settings config schema (type: ignore sequence) needs no change. Agents without the setting behave exactly as before.

Automated tests

New kernel test tests/src/Kernel/PluginBase/AiAgentEntityWrapperRestrictMultipleCallsTest.php covers the four scenarios from the issue summary plus custom-message substitution, using a mock provider that returns fabricated parallel tool calls:

  1. Restricted tool ×2 → nothing executes, both call IDs get tool-role errors, LLM is re-called.
  2. Restricted ×2 + unrestricted ×1 → all three rejected, including the unrestricted call.
  3. Restricted ×1 + unrestricted ×2 → all three execute normally.
  4. Unrestricted ×2 only → unaffected (backward compatibility).
  5. Custom multiple_call_error_message with [tool_name] placeholder is used verbatim.

phpcs (Drupal, DrupalPractice) and PHPStan pass on all changed files.

Manual testing instructions

You need a chat provider that supports parallel tool calls (tested with OpenAI gpt-4o/gpt-5.x) set as default for chat with tools, and the AI Agents Debugger module enabled (drush en ai_agents_debugger) to observe the loop.

1. Form behavior

  1. Edit any agent at /admin/config/ai/agents, expand a tool's General Tool Settings.
  2. Verify the new "Restrict to one call per response" checkbox (unchecked by default) after Require Usage. Checking it reveals the "Multiple call error message" textarea; unchecking hides it.
  3. Check it, save, reload — both values persist (drush config:get ai_agents.ai_agent.<id> tool_settings shows restrict_multiple_calls: 1). Uncheck, save — the message is cleared back to ''.

2. Set up a test agent

  1. Create a new agent with only the List bundles tool enabled.
  2. System prompt: "You are a test agent. When the user asks about bundles of multiple entity types, issue ALL the needed list_bundles tool calls in parallel, in one single response."
  3. In the tool's settings, check Restrict to one call per response, leave the message empty.

3. Violation → all calls rejected → retry

  1. At /admin/config/ai/agents/debug, run the agent with: "In one single response, call list_bundles twice in parallel: once for the entity type 'node' and once for 'taxonomy_term'. Do not call them one after the other."
  2. Expected in loop 1: the assistant response contains two list_bundles calls; neither executes (no tool_started/tool_finished events, no bundle data); the chat history gains one tool-role error message per call ID with the default text ("may only be called once per response… None of the tool calls in this response were executed…").
  3. Expected in loop 2: the LLM is called again with those errors in history and produces a recovery (a single call, or an explanation).
  4. /admin/reports/dblog, type ai_agents: an error entry "The agent … called the single-call tool(s) 'list_bundles' multiple times in one response. All 2 tool calls in the response were rejected."

4. All-or-nothing with a mixed response

  1. Additionally enable List entity types (leave it unrestricted).
  2. Prompt: "In one single response and in parallel: call list_bundles for 'node', call list_bundles for 'taxonomy_term', AND call list_entity_types."
  3. Expected: all three calls rejected in loop 1 — the two list_bundles calls get the default error, and the unrestricted list_entity_types call gets "Error: This tool call was not executed, because the tool(s) 'list_bundles' in the same response may only be called once per response." No entity-type data appears in loop 1.

5. Backward compatibility

  1. Uncheck the restriction, save, re-run the step 3 prompt.
  2. Expected: both list_bundles calls execute in loop 1, real bundle data is returned per call ID, no error text, no new dblog entry — identical to current HEAD behavior.

6. Custom message

  1. Re-check the restriction and set the message to Custom: [tool_name] once only. Save, re-run the step 3 prompt.
  2. Expected: both rejection messages read exactly Custom: list_bundles once only. — confirming the override and the [tool_name] substitution.

Note for reviewers: the retrying LLM only recovers gracefully if the error message reads as an error — that's why the default message is verbose and starts with "Error:". Custom messages should do the same, which the form's field description points out.

Closes #3586051

Merge request reports