Add "throttle API requests" control?
>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3561243. --> Reported by: [ultimike](https://www.drupal.org/user/51132) >>> <p>[Tracker]<br> <strong>Update Summary: </strong>Would a "throttle" control be useful?<br> <strong>Short Description: </strong>Being able to control how quickly API calls are sent to LLMs might help users on lower tiers avoid hitting rate limits.<br> <strong>Check-in Date: </strong>MM/DD/YYYY<br> <em>Metadata is used by the <a href="https://www.drupalstarforge.ai/" title="AI Tracker">AI Tracker.</a> Docs and additional fields <a href="https://www.drupalstarforge.ai/ai-dashboard/docs" title="AI Issue Tracker Documentation">here</a>.</em><br> [/Tracker]</p> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Would it make sense to add some type of "Throttle" control that allows site-builder to limit how many (or how often) LLM API calls are made? While experimenting with AI Agents, I often hit LLM response limits (currently as a Tier 1 user) and it is extremely frustrating. It seems like the chances of hitting a rate limit could be mitigated by throttling the module to only send 1 API request every X seconds (where X is a value set by the site-builder.)</p> <p>I'm not sure if this is the best way of handling this issue, I am aware of <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/ai/issues/3548451" title="Status: Closed (fixed)">#3548451: Get rate limit information normalized</a></span> and the benefit it will likely bring in surfacing better usage information, but maybe it would be helpful to give site-builders some measure of control as well. </p> <p>Honestly, I'm not sure if a throttle control is the best way to handle this issue, but after working with new users to the AI module for a bit, there is definitely some room for improvement. </p>
issue