>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3565806. --> Reported by: [marcus_johansson](https://www.drupal.org/user/385947) >>> <p>[Tracker]<br> <strong>Update Summary: </strong>[One-line status update for stakeholders]<br> <strong>Short Description: </strong>[One-line issue summary for stakeholders]<br> <strong>Check-in Date: </strong>MM/DD/YYYY<br> <em>Metadata is used by the <a href="https://www.drupalstarforge.ai/" title="AI Tracker">AI Tracker.</a> Docs and additional fields <a href="https://www.drupalstarforge.ai/ai-dashboard/docs" title="AI Issue Tracker Documentation">here</a>.</em><br> [/Tracker]</p> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Currently the Automators was setup using pseudo json/prompted json. This means that you basically ask the LLM something like</p> <p><code>Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.\n[{"value": {"state": "the state name"}}]\n</code></p> <p>There are a lot of reasons why its this way, the most important being:</p> <ul> <li>The project stems from the <a href="https://www.drupal.org/project/ai_interpolator">AI Interpolator</a>, that was released more or less when OpenAI released their first GPT-3.5 model. This model did not have structured output.</li> <li>The abstraction layer only started supporting structured output by 1.1.x, and since this will cause (mostly positive) changes in reliability it should really happen on a major version.</li> <li>For the longest time, structured output and tool calling was much slower then pseudo json calling inference - 2-4 times slower.</li> <li>Maybe providers still doesn't support structured output.</li> </ul> <p>Some reasonings why we should use structured output in 2.x.:</p> <ul> <li>It guarantees that if the model can answer in the structure you want, it will.</li> <li>We do not need all our services we currently have to try to extract the json from different potential outputs - we can assume failure or a working json blob.</li> <li>OpenAI and Anthropic at least have the same speed of inference, between pseudo json and structured output. We should try other models, but hopefully this holds true.</li> </ul> <p>Some reasonings why NOT to use structure ouput in 2.x:</p> <ul> <li>It will cause changes in reliability. In almost all cases they are positive, but anyone having a long Automator chain, might actually depend on quirkiness of models for whatever reason.</li> <li>Some models that still exists like GPT-3.5, can do pseudo json, but not structured output. They will no longer work. The most likely problem with this would be local models.</li> </ul> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <p>Decide and write a follow up issue.</p>