Add "throttle API requests" control?
>>> [!note] Migrated issue
<!-- Drupal.org comment -->
<!-- Migrated from issue #3561243. -->
Reported by: [ultimike](https://www.drupal.org/user/51132)
>>>
<p>[Tracker]<br>
<strong>Update Summary: </strong>Would a "throttle" control be useful?<br>
<strong>Short Description: </strong>Being able to control how quickly API calls are sent to LLMs might help users on lower tiers avoid hitting rate limits.<br>
<strong>Check-in Date: </strong>MM/DD/YYYY<br>
<em>Metadata is used by the <a href="https://www.drupalstarforge.ai/" title="AI Tracker">AI Tracker.</a> Docs and additional fields <a href="https://www.drupalstarforge.ai/ai-dashboard/docs" title="AI Issue Tracker Documentation">here</a>.</em><br>
[/Tracker]</p>
<h3 id="summary-problem-motivation">Problem/Motivation</h3>
<p>Would it make sense to add some type of "Throttle" control that allows site-builder to limit how many (or how often) LLM API calls are made? While experimenting with AI Agents, I often hit LLM response limits (currently as a Tier 1 user) and it is extremely frustrating. It seems like the chances of hitting a rate limit could be mitigated by throttling the module to only send 1 API request every X seconds (where X is a value set by the site-builder.)</p>
<p>I'm not sure if this is the best way of handling this issue, I am aware of <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/ai/issues/3548451" title="Status: Closed (fixed)">#3548451: Get rate limit information normalized</a></span> and the benefit it will likely bring in surfacing better usage information, but maybe it would be helpful to give site-builders some measure of control as well. </p>
<p>Honestly, I'm not sure if a throttle control is the best way to handle this issue, but after working with new users to the AI module for a bit, there is definitely some room for improvement. </p>
issue