>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3499610. --> Reported by: [marcus_johansson](https://www.drupal.org/user/385947) >>> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>The Assistants API currently is running different plugins - they can be Agents or RAG (or something in-house). So what happens is that the Assistant API gets a question about something for RAG or instruction to do something for Agents and they action plugin then goes to work and finds information or tells the Assistant it is done.</p> <p>In both those examples the URI's it gives back are deterministic and 100% hallucination free. RAG responses comes from the field in Search API and agents, like the field agent gives back links to field configuration, form configuration etc. of the field it created and edited without using an LLM.</p> <p>The instruction to the assistant then is to rewrite this according to your site goals, in the case of Drupal CMS it is to find where changes happened on the website so the end user can verify them.</p> <p>We had an initial issue that was happening a lot (30% of the time), where the link came back relative to the path, instead of relative to the root. This was fixed in <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/ai/issues/3497916" title="Status: Closed (fixed)">#3497916: AI Deepchat: AI Assistant links are not always relative to root when not absolute</a></span> via code.</p> <p>However other types of hallucination have been seen in issues:<br> <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-3"><a href="https://www.drupal.org/project/ai/issues/3498934" title="Status: Closed (duplicate)">#3498934: Instructions how to use cron are outdated</a></span><br> <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/ai_agents/-/work_items/3498938" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/ai_agents/-/work_items/3498938</a></span><br> <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/ai_agents/-/work_items/3498932" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/ai_agents/-/work_items/3498932</a></span></p> <p>Where it has added a hash or an example.com or similar host to the link, causing the link to fail. Or made up link as an answer to a general issue that doesn't invoke the agents.</p> <h4 id="summary-steps-reproduce">Steps to reproduce</h4> <p>Ask it to create fields or something similar for you and after enough times you might get an hallucinated host or other type of rewrite.</p> <h3 id="summary-proposed-resolution">Possible Solutions</h3> <ul> <li>Make sure that the agent always answers with absolute links</li> <li><span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/ai_agents/-/work_items/3498719" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/ai_agents/-/work_items/3498719</a></span> is one possible solution, but since its still LLM, its not sure it will work.</li> <li>Having a separate link window, where all the action plugins links are shown, similar to how Gemini references links could be used. Meaning you give each link a number and in the LLM that number references the link without any rewrite.</li> <li>Doing the same with a rewrite.</li> <li>Just allowing certain domains to be allowed for absolute link, this will however not work when you use agents that can search and read websites for you.</li> <li>Adding RAG to any question that doesn't involve agents and only let it answer based on this so it won't make up any links - this is already possible to do, but would require underlying vector database in Drupal CMS to be installed or be hosted by DA.</li> </ul> <h3 id="summary-remaining-tasks">Remaining tasks</h3> <p>Research which solutions would work the best.</p>