## Summary Before we ship 1.0, we should confirm that the current rules for choosing which context items reach an AI feature are the right set — and whether anything important is missing. This issue is a **discussion**. The goal is shared understanding of what the module does today, in plain language, so product/UX and engineering can decide together whether to add ranking factors before release. ## Background When someone uses an AI feature on the site, the module picks a small set of **context items** (brand voice, writing rules, section-specific guidance, etc.) and attaches them to the AI’s instructions. There is a size budget: only so many items and so much text can be sent. Selection is **rule-based**, not “the AI decides what’s relevant.” That makes behavior predictable, but it also means many factors compete — and when an item is not chosen, there is usually no visible explanation in the UI today. ## How selection works today (high level) Selection happens in **layers**, from strongest to weakest: 1. **Remove** items that must never be included 2. **Require** items that must always be included 3. **Filter out** items that do not fit the current situation 4. **Group** remaining items into priority tiers 5. **Rank** within the “preference” tier using scope overlap 6. **Apply limits** (item count, token budget) 7. **Attach child items** (subcontext) after parents are chosen If two items are equally strong within a tier, tie-breakers are mostly **most recently updated** (and item ID as a last resort). --- ## Current ranking and selection factors ### 1. Hard exclusions (item is never considered) | Factor | In plain language | |--------|-------------------| | **Not published** | Draft or unpublished items are ignored. | | **No view permission** | If the current user cannot view the item, it is skipped. | | **Never include** | Per AI feature/agent setting: specific items are blocked even if they would otherwise match. | | **Context filters** | If an item says it only applies in certain situations, and the current situation does not match, the item is removed. Examples: wrong language, wrong site section/URL, wrong content type, or wrong specific page/entity. | | **Broken parent link** | A child item whose parent is missing is not used on the “always include” path. | ### 2. Guaranteed inclusion tiers (these beat normal ranking) These are applied **in this order**. Earlier tiers win when the total item limit is reached. | Tier | In plain language | |------|-------------------| | **Global items** | Marked “apply everywhere.” Included for every AI feature, up to a **global item limit** (site default: 3). If there are more global items than the limit allows, **most recently updated** global items win. | | **Always include** | Per AI feature/agent setting: chosen items are forced in, regardless of topic matching. | | **Auto-included by situation** | Items tied to **what is being worked on right now** — especially **specific target content** (e.g. “this guidance is for Article #42”). These skip subscription ranking because the situation itself selects them. | | **Topic-matched items (scored)** | Everything else that survived filters and overlaps with the AI feature’s **scope subscriptions** (topics/preferences). Sorted by relevance score (see below). | ### 3. Scope subscriptions (the main “preference” ranking) Each AI feature can subscribe to topics like: - **Use case** — e.g. “Working with text,” “Working with Canvas” - **Tags** — free-form labels from the AI Context Tags vocabulary - **Language** - e.g. Spanish, Arabic - **Site section** — URL/path areas - **Entity bundle** — content types such as “Article” or “Basic page” **How overlap is scored:** - More overlap with the AI feature’s subscriptions → higher score → more likely to be chosen. - If the item has **no value** for a subscribed topic, it is treated as **broad/generic** for that topic and gets full credit (not penalized). - If the AI feature subscribes to several values under one topic (e.g. two tags), matching **more of them** helps more than matching just one. - Topics have different **importance weights** in the math. Roughly, from most to least influential in scoring today: **Entity bundle → Site section → Use case / Tag → Language**. (Exact weights are internal; the relative order matters more for this discussion.) **Important UX nuance:** Language, site section, and entity bundle act as **both** filters (must match current situation) **and** preferences (boost ranking when the AI feature subscribes to them). Use case and tags are **preferences only** — they do not auto-filter by current page context. ### 4. Selection mode (when the AI feature has no topic subscriptions) | Mode | In plain language | |------|-------------------| | **Minimal** (default) | Safe default: only global items, always-included items, and situation-based auto-includes. Does **not** pull in general topic-scoped items just because they exist. | | **Match all** | Broader recall: if there are no topic subscriptions, many published items can be considered, ordered by **most recently updated**, still subject to item and token limits. Intended for cases that intentionally need wide coverage. | ### 5. Limits and final cutoffs | Factor | In plain language | |--------|-------------------| | **Max context items** | Cap on how many items are selected (site default: 10; per AI feature can override). | | **Max global items** | Separate cap on global items (site default: 3). | | **Max tokens** | Cap on total rendered text sent to the AI (site default: 1200 tokens). Items lower in the final order may be dropped or shortened when the budget runs out. | | **Language translation** | Items are shown in the current content language when a translation exists. | ### 6. Subcontext (parent/child items) After the main list is built: - **Required children** are always added with their parent. - **Conditional children** may be chosen by a separate AI decision (parent content + child purpose + user task), when that feature is enabled. - Children appear **immediately after their parent** in the final output. - Subcontext resolution happens **after** the main item cap — so children can add extra items beyond the primary max-items limit. ### 7. Things that are **not** ranking factors today Worth calling out for the 1.0 discussion: | Not a factor | Notes | |--------------|-------| | **How well the item matches the user’s actual task/prompt** | Selection does not read the user’s message for relevance (except conditional subcontext decisions). | | **Item title, length, or author** | Not used in ranking. | | **Usage/popularity** | Frequently used items are not boosted. | | **Recency (except tie-breakers)** | “Recently updated” only breaks ties within global items and match-all mode — not the main subscription score. | | **Manual “assigned items” on an agent** | Mentioned in docs, but **not implemented** in code. The working equivalent today is **Always include**. | | **“Always include when relevant” on the item itself** | Discussed/planned in UX docs; **not implemented** as an item-level switch yet. | --- ## Discussion questions for 1.0 1. **Is the four-tier priority order right?** Global → Always include → Situation auto-include → Topic score. 2. **Are the current subscription topics enough?** Use case, tags, language, site section, entity bundle — or do we need item-level priority, task-aware ranking, recency boosts, etc.? 3. **Should broad/generic items (no topic labels) rank higher or lower?** Today they score well on subscribed topics because they are treated as universally applicable for that topic. 4. **Do we need visible “why was / wasn’t this item included?” before 1.0?** Not a ranking factor itself, but likely the biggest source of confusion when ranking excludes an item. 5. **Should item-level “always include when filters match” ship in 1.0?** Already identified in UX planning as a way to reduce reliance on two-sided subscription matching. 6. **Is match-all mode too risky as an escape hatch for small sites?** Minimal mode is safer but requires understanding subscriptions or always-include overrides. --- ## Acceptance criteria (for this discussion issue) - [ ] Product/UX and engineering agree the list above accurately reflects current behavior - [ ] Open gaps for 1.0 are identified (must-have vs nice-to-have vs post-1.0) - [ ] Follow-up issues created for any agreed changes ## AI disclosure - [x] AI Assisted Issue