Running into Token limits when using the API
>>> [!note] Migrated issue
<!-- Drupal.org comment -->
<!-- Migrated from issue #3537369. -->
Reported by: [sirclickalot](https://www.drupal.org/user/403880)
>>>
<p>I am trying to use an Automator to produce a chronological report of students progress.</p>
<p>I am using Drupal and I have made a simple content type called 'Student log'.</p>
<p>Each Student log has a multivalued Paragraphs Revisions field called 'Log items' - - machine name: <code>field_session_logs</code>.</p>
<p>Each Log item is basically a<strong> Long text field</strong> - machine name: <code>field_session_coverage</code> 'though that is pretty irrelevant' since it is the <code>field_session_logs</code> that we feed to the LLM as you willsee below.</p>
<p>In summary, <strong>the text</strong> that I want the LLM to analyse is<strong> spread across multiple paragraphs</strong> in this model...</p>
<pre>Teacher log (teachers_session_log) <- Content type<br> |- Session logs (field_session_logs) <- Paragraphs revision field (multivalued)<br> |- Session log item (teachers_session_log_item) <- Paragraph<br> |- Notes (field_session_coverage) <- Text (formatted, long)</pre><p>So essentially, what I am trying to get the LLM to do is to<strong> iterate through</strong> each of the <strong>Notes </strong>fields in each Paragraph and to build up a block of text to analyse.</p>
<p>I have a field containing analytical instructions and that field is called 'Analytical rubric' - machine name: <code>field_analytical_rubric</code>.</p>
<p>The 'Analytical rubric' field typically contains...</p>
<blockquote><p>
Analyse it in dated order.<br>
Analyse it in terms of:<br>
1) What was covered.<br>
2) Delivery of homework and other assignments by the student.<br>
3) Misunderstandings and misgivings and related progress in overcoming these.<br>
4) Progress with repeated topics over time.
</p></blockquote>
<p>I have set up Long text field to <strong>receive the results</strong> back from OpenAI...</p>
<p><img src="https://www.drupal.org/files/issues/2025-07-22/ai-module-automator-setup.png" alt="/ai-module-automator-setup"></p>
<p>Just for extra clarity, the Automator prompt is...<br><br>
<code>Follow the instructions in [node:field_analytical_rubric] to assess the content of [node:field_session_logs].</code>.</p>
<p>The complaint from ChatGPT is...</p>
<p><code>"Request too large for gpt-4o in organization org-oU8jWrPmkVR1FMmn9ZpETOsX on tokens per min (TPM): Limit 30000, Requested 44501. The input or output tokens must be reduced in order to run successfully."</code></p>
<p>I strongly suspect that my expectation that the the Automator will consume each <code>teachers_session_log_item</code> <strong>Paragraph</strong> and know to pull out just what we want which is the text from the <code>field_session_coverage</code> field is probably hopelessly naive but I am just not sure how to express that with tokens. </p>
<p>In my tests, the Paragraphs Revisions field called 'Log items' contains <strong>44 separate paragraphs</strong> for the LLM to get through but in reality that's a about half as many as there would be in any given year so it's got to be able to deal with that many.</p>
<p>So, I have been trying to find out how downgrade my request to GPT3.5x instead and I have changed theses two settings, but it's all a bit of guesswork.</p>
<p>I've tried...</p>
<p><img src="https://www.drupal.org/files/issues/2025-07-22/ai-module-default-provider-chat.png" alt="ai-module-default-provider-chat"></p>
<p>and...</p>
<p><img src="https://www.drupal.org/files/issues/2025-07-22/ai-module-default-provider-chat-complex-json.png" alt="ai-module-default-provider-chat-complex-json"></p>
<p>Even so, after several cache clears just in case, I am still seeing exactly the same complaint back from OpenAI; my settings changes appear to have done nothing as it is still trying to use GPT4o.</p>
<p>Does anyone have any ideas or other thoughts they may hep steer?</p>
issue