>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3582364. --> Reported by: [marcus_johansson](https://www.drupal.org/user/385947) Related to !1381 >>> <p>[Tracker]<br> <strong>Update Summary: </strong>[One-line status update for stakeholders]<br> <strong>Short Description: </strong>Add documentation for the Text Chunker service<br> <strong>Check-in Date: </strong>MM/DD/YYYY<br> [/Tracker]</p> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>The <code>ai.text_chunker</code> service (<code>src/Utility/TextChunker.php</code>) provides NLP-friendly text chunking with configurable token-based chunk sizes and overlap. It is used by the AI Search module for splitting documents into chunks for embedding, but is a general-purpose public service that any module can use.</p> <p>The service works with the <code>ai.tokenizer</code> service to split text into chunks of a maximum token size with a configurable minimum overlap between consecutive chunks. The overlap ensures that context is not lost at chunk boundaries, which is important for retrieval-augmented generation (RAG) and search use cases.</p> <p>There is currently no documentation for this service.</p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <ul> <li>Create a documentation page at <code>docs/developers/text_chunker.md</code> documenting the <code>ai.text_chunker</code> service.</li> <li>Document the key method: <code>chunkText(string $text, int $maxSize, int $minOverlap): array</code> and <code>setModel(string $model)</code>.</li> <li>Explain the overlap mechanism and why it matters for RAG/embedding use cases.</li> <li>Include a usage example showing how to inject the service, set a model, and chunk a document.</li> <li>Reference the relationship with the <code>ai.tokenizer</code> service.</li> <li>Add the new page to the <code>mkdocs.yml</code> nav under the Develop section (e.g. under Helper Services).</li> </ul> <h3 id="summary-ai-usage">AI usage (if applicable)</h3> <p>[x] AI Assisted Issue<br> This issue was generated with AI assistance, but was reviewed and refined by the creator.</p> <p>[ ] AI Assisted Code<br> [ ] AI Generated Code<br> [ ] Vibe Coded</p> <p>- <strong>This issue was created with the help of AI</strong></p>