Prompt ignored in vision plugin
>>> [!note] Migrated issue
<!-- Drupal.org comment -->
<!-- Migrated from issue #3515986. -->
Reported by: [mrdalesmith](https://www.drupal.org/user/2612656)
Related to !119
>>>
<h3 id="summary-problem-motivation">Problem/Motivation</h3>
<p>The definition of the Drupal\ai_agents\Plugin\AiFunctionCall\Vision functioon call allows the provision of a prompt string, but the code in the execute() method immediately overrides it:</p>
<div class="codeblock">
<pre><span style="color: #000000"><span style="color: #0000BB"><?php<br> </span><span style="color: #FF8000">// Collect the context values.<br> </span><span style="color: #0000BB">$prompt </span><span style="color: #007700">= </span><span style="color: #0000BB">$this</span><span style="color: #007700">-></span><span style="color: #0000BB">getContextValue</span><span style="color: #007700">(</span><span style="color: #DD0000">'prompt'</span><span style="color: #007700">);<br> </span><span style="color: #0000BB">$image_id </span><span style="color: #007700">= </span><span style="color: #0000BB">$this</span><span style="color: #007700">-></span><span style="color: #0000BB">getContextValue</span><span style="color: #007700">(</span><span style="color: #DD0000">'image_id'</span><span style="color: #007700">);<br> </span><span style="color: #0000BB">$default </span><span style="color: #007700">= </span><span style="color: #0000BB">$this</span><span style="color: #007700">-></span><span style="color: #0000BB">aiProviderManager</span><span style="color: #007700">-></span><span style="color: #0000BB">getDefaultProviderForOperationType</span><span style="color: #007700">(</span><span style="color: #DD0000">'chat_with_image_vision'</span><span style="color: #007700">);<br> </span><span style="color: #0000BB">$provider </span><span style="color: #007700">= </span><span style="color: #0000BB">$this</span><span style="color: #007700">-></span><span style="color: #0000BB">aiProviderManager</span><span style="color: #007700">-></span><span style="color: #0000BB">createInstance</span><span style="color: #007700">(</span><span style="color: #0000BB">$default</span><span style="color: #007700">[</span><span style="color: #DD0000">'provider_id'</span><span style="color: #007700">]);<br> </span><span style="color: #0000BB">$prompt </span><span style="color: #007700">= </span><span style="color: #DD0000">"Could you describe this image?\n"</span><span style="color: #007700">;<br></span><span style="color: #0000BB">?></span></span></pre></div>
<h3 id="summary-proposed-resolution">Proposed resolution</h3>
<p>Either the prompt in the definition should be removed and the hardcoded prompt relied on, or else the prompt from the definition should be used and only overwritten if it is not set.</p>
issue