>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3548451. --> Reported by: [marcus_johansson](https://www.drupal.org/user/385947) Related to !990 >>> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Some of the providers support returning limit information in the response of for example chat requests. </p> <p>See for example:<br> <a href="https://platform.openai.com/docs/guides/rate-limits#rate-limits-in-headers">https://platform.openai.com/docs/guides/rate-limits#rate-limits-in-headers</a><br> <a href="https://docs.claude.com/en/api/rate-limits#response-headers">https://docs.claude.com/en/api/rate-limits#response-headers</a></p> <p>For the OpenAI based clients, ince we inject a Guzzle client by default, we should be able to use a Middleware to collect the header data and use it.</p> <p>We should also add this to our OpenAI default provider with a default open ai solution, and then add a method that providers can use to set this, depending on headers used.</p> <p>We will start with none-streamed Chat.</p> <h3 id="summary-remaining-tasks">Remaining tasks</h3> <ul> <li>Create a DTO called ChatProviderLimitsDto.</li> <li>Use the DtoBaseMethodsTrait</li> <li>Add a constructor to the ChatProviderLimitsDto with RateLimitMaxRequests (int), RateLimitMaxTokens (int), RateLimitRemaingRequests (int), RateLimitRemainingTokens (int), RateLimitResetRequests (int seconds), RateLimitResetTokens (int seconds). All optional and nullable.</li> <li>In the ChatOutput add a getter and setter for RateLimits, like getRateLimits and setRateLimits that gets and sets the ProviderLimitsDto. Instantiate with a ChatProviderLimitsDto, with all values nulled.</li> <li>In the OpenAiBasedProviderClientBase add a Middleware for Guzzle that takes the response and writes the headers to a variable.</li> <li>In the OpenAiBasedProviderClientBase create a method called setChatRateLimits, that maps the variables to a constructed ChatProviderLimitsDto object. The method returns the object or null. Assume OpenAI, but do isset around each key, so it works with none OpenAI providers.</li> <li>In the OpenAiBasedProviderClientBase after the none-streamed response invoke this method and look if an object was given back and set this on the ChatOutput.</li> <li>Documented this as well in docs/developers/writing_an_ai_provider.md and under docs/developers/call_chat.md</li> </ul>