Issue #3586315: Avoid full catalog scan when agent has no scope subscriptions.

Pravesh Pooniarequested to merge
issue/ai_context-3586315:3586315-no-subscription-prefilter into 1.0.x

Description

AiContextSelector::getPrefilteredPublishedItemIds() returned NULL when $scopeSubscriptions was empty, causing the chunk loop to fall back to a full catalog scan on every AI request.

For SELECTION_MODE_MINIMAL with no subscriptions, only global items and auto-include candidates can ever be selected. Subscription-scoped items are always discarded during scoring, so loading every published context item is unnecessary. On sites with many context items, this results in dozens of extra chunked queries per request, affecting the common "global items only" agent configuration.

This MR replaces the full scan with a targeted prefilter when using SELECTION_MODE_MINIMAL with no subscriptions.

The prefilter candidate set is the union of:

  • Global items — retrieved from the scope index (scope_id = 'global').
  • Unindexed items — published items with no scope index rows (target entity-only items, items with no scope, etc.). Looked up via a LEFT JOIN anti-join on the entity data table, intersected with access-checked published IDs.
  • Current target-entity items — items that explicitly reference the current target entity via the DER field. Covers the edge case where an item has both a subscription scope (indexed) and a target entity reference.
  • Custom non-subscription scope items — items indexed under enabled scope plugins where supportsSubscriptions() is FALSE (excluding global and target_entity, which are handled separately).

SELECTION_MODE_MATCH_ALL is unchanged and continues to scan the full catalog.

Files changed

  • AiContextScopeIndexService
    • Added getPublishedItemIdsWithoutScopeIndex() — anti-join lookup for published default-translation items absent from the scope index. Falls back to all published IDs when the index table is missing.
    • Added getItemIdsByScopeIds() — indexed lookup by scope plugin ID.
    • Injects @entity_type.manager to resolve the entity data table name.
  • AiContextScopeResolver
    • Added getUnindexedItemIds() — intersects anti-join results with access-checked published IDs.
    • Added getNonSubscriptionIndexedItemIds() — collects items indexed under custom non-subscription scope plugins.
  • AiContextSelector
    • Added buildNoSubscriptionCandidateIds() and loadTargetEntityCandidateIds().
    • Updated getPrefilteredPublishedItemIds() to accept $selectionMode (before $debugLogging) and apply the targeted prefilter for SELECTION_MODE_MINIMAL when no subscriptions are present.
    • Debug log uses @target_entity placeholder (not @te).
  • AiContextSelectorNoSubscriptionPrefilterTest
    • New kernel test class (8 tests) covering global selection, MATCH_ALL, neverInclude, alwaysInclude, target-entity auto-include, custom non-subscription scopes, and debug-log prefilter narrowing.
  • ai_context_test_scope_auto_include
    • Test module with a non-subscription scope plugin for extensibility coverage.

Testing instructions

  1. Enable the module and create several context items:

    • at least one with Global scope
    • one or more with a non-global scope (for example, Language or Tag)

  2. Create an agent at /admin/config/ai/agents and leave its scope subscriptions empty.

  3. Enable debug logging at Configuration → AI Context → Settings → General → Enable debug logging.

  4. Trigger an AI request using that agent (for example, through the AI Agents Debugger).

  5. Open the Drupal log (/admin/reports/dblog) and verify a message similar to:

    AiContextSelector: no-subscription prefilter selected X of Y published item IDs (global: …, unindexed: …, target-entity: …)

    where X is significantly smaller than Y (only prefilter candidates are selected rather than the full catalog).

  6. Verify that the agent response still includes the global context item.

  7. Repeat with an agent that has scope subscriptions and verify behavior remains unchanged.

  8. Run kernel tests:

    vendor/bin/phpunit --filter AiContextSelectorNoSubscriptionPrefilterTest

Checklist

  • I have updated the MR title to use format: Issue #1234: My issue title.
  • I have updated the MR description to include: Closes #1234
  • I have performed a self-review of my own code
  • I have added or updated tests, or explained in the description why this change is not covered by tests
  • I have updated documentation for any new or changed functionality
  • I have written testing instructions and verified them locally
  • I have noted any required post-merge steps (config imports, cache rebuilds, manual changes)
  • This MR contains no breaking API or hook changes, or they are explicitly documented in the description

AI Compliance

Note

Check the one that best describes your usage, or leave all unchecked if AI was not significantly used.

  • AI Assisted Code
    Mainly written by a human; AI used for autocomplete or partial generation under full human supervision.

  • AI Generated Code
    Mainly generated by AI, reviewed and approved by a human before this MR was created.

  • Vibe Coded
    Generated by AI and only functionally reviewed before this MR was created.

Closes #3586315 (closed)

Edited by Kristen Pol

Merge request reports