Add optional RAW vectors in RAG results
>>> [!note] Migrated issue
<!-- Drupal.org comment -->
<!-- Migrated from issue #3525296. -->
Reported by: [nikro](https://www.drupal.org/user/576416)
Related to !618
>>>
<p>--- AI TRACKER METADATA ---<br>
<strong>Update Summary: </strong>Needs to fix Scott’s feedback.<br>
<strong>Check-in Date: </strong>MM/DD/YYYY (US format) [When we should see progress/get an update]<br>
<strong>Due Date:</strong> MM/DD/YYYY (US format) [When the issue should be fully completed]<br>
<strong>Blocked by:</strong> [#XXXXXX] (New issues on new lines)<br>
<strong>Additional Collaborators:</strong> @username1, @username2<br>
AI Tracker found here: <a href="https://www.drupalstarforge.ai/" title="AI Tracker">https://www.drupalstarforge.ai/</a><br>
--- END METADATA ---</p>
<h2>Problem / Motivation</h2>
<p>
The AI VDB provider plugin manager currently discovers embedding providers but does not expose which field is used to store raw embeddings (at least I didn't find it). As a result, Search API AI backends cannot conditionally include raw vectors in their output, leading to fatal errors when a backend expects a vector field that the provider did not return.
</p>
<h3>Proposed resolution</h3>
<p></p><ol>
<li>Add an optional method <code>getRawEmbeddingFieldName()</code> to the VDB provider plugin interface + add basic implementation in the abstract class</li>
<li>If implemented, this method returns the name of the field containing raw embeddings; otherwise it returns <code>NULL</code>.</li>
<li>Extend the Search API AI search backend UI with a checkbox “Include raw embedding vector” (default off).</li>
<li>When checked and the provider implements <code>getRawEmbeddingFieldName()</code>, include that field in the search result output.</li>
</ol>
<h3>User interface changes</h3>
<ul>
<li>
In the Search API AI search backend settings form, add a checkbox:<br>
<label><input type="checkbox" name="include_raw_embeddings"> Include raw embeddings</label>.
</li>
<li>
Provide descriptive help text: “When enabled and supported by your VDB provider, raw embedding vectors will be returned in each result.”
</li>
</ul>
issue