>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3552816. --> Reported by: [david bishop](https://www.drupal.org/user/1163950) >>> <p>--- AI TRACKER METADATA ---<br> Update Summary: Discuss AI Failover<br> Check-in Date: MM/DD/YYYY (US format) [When we should see progress/get an update]<br> Due Date: MM/DD/YYYY (US format) [When the issue should be fully completed]<br> Blocked by: [#XXXXXX] (New issues on new lines)<br> Additional Collaborators: @username1, @username2<br> AI Tracker found here: <a href="https://www.drupalstarforge.ai/">https://www.drupalstarforge.ai/</a><br> --- END METADATA ---</p> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Currently, we do not offer any failover service for providers. This means that if your provider is calling OpenAI and they are down or you run into rate-limiting issues, we need to be able to fail over to a secondary (or third) provider.</p> <p>There are various reasons why a failover could happen or be desired<br> - Credit has run out<br> - Time limit (OpenAI has a rate limit of 160 calls every 3 hours on certain models on the $20 monthly subscription)<br> - Message is too large for the current model, particularly when working with local LLMs</p> <p>An issue will be added for advanced load balancing, which should likely be external to the AI core.</p> <p>But this issue has simple logic, if provider one fails with one of AIBadRequestException, AIRequestErrorException, AIQuotaException, AIResponseException, AIRateLimitException, AIAccessDeniedEXception then it retries with the next in line until it fails on all.<br> This should first have <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/ai/issues/3552774" title="Status: Closed (fixed)">#3552774: Create linkable schema for providers</a></span> in place, before starting work on this.</p> <p>The discussion here is:<br> 1. How does the UI/UX looks like.<br> 2. How do we handle stateful calls like OpenAI Chat call with Files.<br> 3. How do we store this, should the failover be set on the default and then on each submodule, or should we have a failover model that always runs when something fails.<br> 4. How does this tie into other useful parameters that could be stored against providers such as known rate limit or budget<br> 5. Should it maintain context for models to know (I've hit the 3 hour limit for Provider X but I'll come back later)</p> <h4 id="summary-steps-reproduce">Steps to reproduce (required for bugs, but not feature requests)</h4> <p>Please provide information like AI modules enabled, which AI provider, browser, etc.</p> <p>Rate Limit Test<br> 1. Create an OpenAI account<br> 2. Set up a Drupal OpenAI Provider and add the API Key<br> 3. Use the Chat Generation Explorer to call GPT-5 which currently has a rate limit on the free account of 10 messages every 5 hours.</p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <p>This is just brainstorming:<br> 1. We create failover settings in default settings for now.<br> 2. The <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/ai/issues/3552774" title="Status: Closed (fixed)">#3552774: Create linkable schema for providers</a></span> should .<br> 3. The ability to select a failover provider for each provider, this would enable them to be chained together.</p> <h3 id="summary-remaining-tasks">Remaining tasks</h3> <h3>Optional: Other details as applicable (e.g., User interface changes, API changes, Data model changes)</h3>