[Discuss] Optimize context injection for token limits and agent comprehension (#3574925) · Issues · project / ai_context

[Discuss] Optimize context injection for token limits and agent comprehension

>>> [!note] Migrated issue   Reported by: [kristen pol](https://www.drupal.org/user/8389) >>> [Tracker] Update Summary: [One-line status update for stakeholders] Check-in Date: MM/DD/YYYY Blocked by: [#XXXXXX] (New issues on new lines) Additional Collaborators: @username1, @username2 Metadata is used by the <a href="https://www.drupalstarforge.ai/" title="AI Tracker">AI Tracker.</a> Docs and additional fields <a href="https://www.drupalstarforge.ai/ai-dashboard/docs" title="AI Issue Tracker Documentation">here</a>. [/Tracker] <h3 id="summary-problem-motivation">Problem/Motivation</h3> Right now, context is injected into end of the prompt without much structure. It's the ID, tags (outdated), and content. It used to have title but then that sometimes caused issues so was removed but that was for TEST content so it probably should be readded. <h3 id="summary-proposed-resolution">Proposed resolution</h3> Some ideas from ChatGPT below. Discuss and figure out next steps. At minimum, we should provide the label, purpose, and scope. In the future, we should probably add a summarization of the context. <h3 id="summary-remaining-tasks">Target date or deadline</h3> February or March 2026 <h3 id="summary-remaining-tasks">Remaining tasks</h3> <ul> <li>Discuss</li> <li>Figure out next steps</li> </ul> <h3>ChatGPT's analysis</h3> If the tool is returning raw markdown for brand guidelines / writing standards / governance, the main problems you’ll hit are: <ul> <li>too many tokens (cost + context window pressure)</li> <li>repetition (same “always include” stuff every call)</li> <li>low signal-to-noise (agents skim; they don’t “feel” what’s important unless you structure it)</li> </ul> Here are practical ways to “compress” without losing meaning—and still keep it usable as context. 1) Return a structured “context pack” instead of one big markdown blob Even if the content inside is markdown, wrap it in a predictable structure so the agent can use it: <ul> <li>items[]: each item has id, title, type, priority, excerpt, content_markdown</li> <li>summary: a short synthesized “rules at a glance”</li> <li>token_budget_hint: what to do if it’s too long (e.g., drop low priority items)</li> </ul> This helps the agent decide what to read first and what to ignore. 2) Add a two-tier output: “compressed” + “full” For guidelines especially, you usually want: Tier A: compressed “rule sheet” (fast to read) <ul> <li>10–30 bullet rules max</li> <li>grouped by category: Voice, Terms, Legal/Gov, Formatting, Accessibility, SEO, etc.</li> <li>include “Do/Don’t” and examples only when they’re truly essential</li> </ul> Tier B: full markdown (only when needed) Return full markdown for the top N items, and for the rest return an excerpt + a pointer (“expand if needed”). You already have max_context_items; add a second knob like: <ul> <li>verbosity: compressed | normal | full</li> </ul> or <ul> <li>include_full_text: boolean</li> </ul> Default it to compressed for agents. 3) Compress within each item using a stable recipe For brand/writing standards, the best compression isn’t “summarize everything” — it’s extract the actionable constraints. A great template per item: <ul> <li>Purpose (1 line)</li> <li>Rules (bullets, imperative)</li> <li>Must / Must not (hard constraints)</li> <li>Preferred terms / Avoid terms</li> <li>Examples (only 1–2)</li> <li>When unsure (fallback rule)</li> </ul> That turns a 2-page guideline into something an agent can reliably follow. 4) De-duplicate globally with a “common rules” section If you have always-included items like “Brand voice” and “Writing standards,” they’ll appear constantly. Instead: <ul> <li>create a Common Rules section once at the top (compressed)</li> <li>then for each item, include only the delta (“what’s unique here”)</li> </ul> This is huge for token savings. 5) Add “compression metadata” so you can tune it later Even if the agent only needs markdown, your selector should return metadata for ranking + trimming: <ul> <li>importance (hard/soft)</li> <li>scope (global | per-agent | per-content-type | per-entity)</li> <li>freshness (last updated)</li> <li>length_chars or approximate token estimate</li> </ul> Then you can enforce budgets: <ul> <li>always include all importance=hard</li> <li>fit as many soft items as possible under a token cap</li> </ul> 6) Consider a dedicated “Guideline Digest” cache Because these are stable documents, you can precompute: <ul> <li>a compressed digest per item</li> <li>a compressed digest per pool/agent</li> </ul> Then runtime tool calls return the digest instantly, and only fetch full text when asked. Practical: store digest in the entity (field) or computed storage + cache tags. 7) A simple, agent-friendly markdown format If you want to keep it markdown-only, I’d output like this: <pre>## Context Pack (compressed) **Task:** <task> **Selected items:** 12, 44, 51 ### Rules to follow (highest priority) - Must: ... - Must not: ... - Prefer: ... - Avoid: ... ### Item 12 — Brand Voice (hard) - Summary: ... - Do: ... - Don’t: ... - Preferred terms: ... - Avoid terms: ... ### Item 44 — Accessibility & Inclusive language (hard) - Summary: ... ...</pre>That’s still “markdown,” but it’s compressed and scannable. 8) One immediate code-level tweak you can do without changing architecture Right now you’re building: <code>"Selected IDs ...\n\n" . $result['text']</code> Instead, generate: <ul> <li>a short header</li> <li>then bulletized, categorized content</li> <li>limit each item to a max character budget unless verbosity=full</li> </ul> Even crude limits help a lot. What I would implement first (smallest change, biggest win) Add an optional input: verbosity (compressed default) Return: <ul> <li>a short “Rules to follow” section (global digest)</li> <li>per-item “Summary + Key Rules” only</li> <li>include full markdown only when verbosity=full</li> </ul> If you paste a sample of what $result['text'] looks like today (even one item), I can rewrite the output format into a compressed version that still preserves the important constraints. <h3 id="summary-ai-usage">AI usage (if applicable)</h3> [x] AI Assisted Issue This issue was generated with AI assistance, but was reviewed and refined by the creator. [ ] AI Assisted Code This code was mainly generated by a human, with AI autocompleting or parts AI generated, but under full human supervision. [ ] AI Generated Code This code was mainly generated by an AI with human guidance, and reviewed, tested, and refined by a human. [ ] Vibe Coded This code was generated by an AI and has only been functionally tested.

issue