Add optional RAW vectors in RAG results
>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3525296. --> Reported by: [nikro](https://www.drupal.org/user/576416) Related to !618 >>> <p>--- AI TRACKER METADATA ---<br> <strong>Update Summary: </strong>Needs to fix Scott&rsquo;s feedback.<br> <strong>Check-in Date: </strong>MM/DD/YYYY (US format) [When we should see progress/get an update]<br> <strong>Due Date:</strong> MM/DD/YYYY (US format) [When the issue should be fully completed]<br> <strong>Blocked by:</strong> [#XXXXXX] (New issues on new lines)<br> <strong>Additional Collaborators:</strong> @username1, @username2<br> AI Tracker found here: <a href="https://www.drupalstarforge.ai/" title="AI Tracker">https://www.drupalstarforge.ai/</a><br> --- END METADATA ---</p> <h2>Problem / Motivation</h2> <p> The AI VDB provider plugin manager currently discovers embedding providers but does not expose which field is used to store raw embeddings (at least I didn't find it). As a result, Search API AI backends cannot conditionally include raw vectors in their output, leading to fatal errors when a backend expects a vector field that the provider did not return. </p> <h3>Proposed resolution</h3> <p></p><ol> <li>Add an optional method <code>getRawEmbeddingFieldName()</code> to the VDB provider plugin interface + add basic implementation in the abstract class</li> <li>If implemented, this method returns the name of the field containing raw embeddings; otherwise it returns <code>NULL</code>.</li> <li>Extend the Search API AI search backend UI with a checkbox &ldquo;Include raw embedding vector&rdquo; (default off).</li> <li>When checked and the provider implements <code>getRawEmbeddingFieldName()</code>, include that field in the search result output.</li> </ol> <h3>User interface changes</h3> <ul> <li> In the Search API AI search backend settings form, add a checkbox:<br> <label><input type="checkbox" name="include_raw_embeddings"> Include raw embeddings</label>. </li> <li> Provide descriptive help text: &ldquo;When enabled and supported by your VDB provider, raw embedding vectors will be returned in each result.&rdquo; </li> </ul>
issue