AI Search: Decide where to manage configuration for Search API
>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3456118. --> Reported by: [scott_euser](https://www.drupal.org/user/3267594) >>> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Looking at search_api_ai, the embeddings are added as a data type, but the rest of the content indexes as usual via e.g. database, SOLR, etc.</p> <p>Allowing this combination is probably still right as we can allow users to do a combination of exposed filters &amp; vector database search, e.g. asking a question and caring only about results from blog post content type. Thoughts?</p> <p>I can imagine a situation where the Vector Storage Plugin selected e.g. is SOLR and the search index is SOLR. The Vector Storage Plugin for SOLR can require selection of a search api solr index and then just grab configuration from that for example. </p> <p>Or similarly the Vector Storage Plugin can be Pinecone, yet the standard indexing done into SOLR. The user then queries some combination of the SOLR index and embeddings. In this scenario however it gets complicated:</p> <ol> <li>Do we retrieve the topK embeddings first, then apply that as a condition e.g. to the SOLR query</li> <li>Do we run the SOLR query first, then apply that as a meta data filter into the vector storage database</li> </ol> <p>I think it must be (1) otherwise each exposed filter option would need to be added as metadata as e.g. with Pinecone filtering by meta data is quite limited (<a href="https://docs.pinecone.io/guides/data/filter-with-metadata#querying-an-index-with-metadata-filters">https://docs.pinecone.io/guides/data/filter-with-metadata#querying-an-index-with-metadata-filters</a>)</p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <p>?</p>
issue