Implement Advanced Input Mode with Token Chunking for Text Automator
>>> [!note] Migrated issue
<!-- Drupal.org comment -->
<!-- Migrated from issue #3489273. -->
Reported by: [seogow](https://www.drupal.org/user/3065779)
Related to !524
>>>
<h3 id="summary-problem-motivation">Problem/Motivation</h3>
<p>The current Text Automator in the AI module effectively handles many-to-many relationships using prompts like “Provide output row per each input row.” However, it encounters difficulties when processing long texts due to token limitations of Large Language Models (LLMs).</p>
<h3 id="summary-proposed-resolution">Proposed resolution</h3>
<p>Introduce an “Advanced Mode (Token, Chunked)” option under the “Automator Input Mode” settings. This mode will:</p>
<ol>
<li>Allow users to define the number of tokens per chunk, enabling customization based on specific LLM constraints and content requirements.</li>
<li>Divide lengthy input texts into manageable chunks according to the user-defined token count.</li>
<li>Sequentially process each chunk through the LLM with instructions to iteratively refine the output as more data is provided.</li>
<li>Aggregate the refined outputs to form a cohesive final result.</li>
</ol>
<h3 id="summary-remaining-tasks">Remaining tasks</h3>
<ul>
<li>Develop the chunking mechanism to split input texts based on user-defined token counts.</li>
<li>Implement logic to manage iterative refinement across chunks.</li>
</ul>
<h3 id="summary-ui-changes">User interface changes</h3>
<ul>
<li>Add the “Advanced Mode (Token, Chunked)” option to the “Automator Input Mode” dropdown menu within the Text Automator settings.</li>
<li>Include a field for users to specify the desired number of tokens per chunk.</li>
</ul>
issue