>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3582953. --> Reported by: [webchick](https://www.drupal.org/user/24967) Related to !2 >>> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>Now that we have an evals framework as of <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/ai_best_practices/-/work_items/3581832" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/ai_best_practices/-/work_items/3581832</a></span> let's document how to use it in practice.</p> <h3>Scenario 1: Before / after new rule</h3> <p><span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/ai_best_practices/-/work_items/3581669" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/ai_best_practices/-/work_items/3581669</a></span> added a "stub" skill for documentation but <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/ai_best_practices/-/work_items/3581687" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/ai_best_practices/-/work_items/3581687</a></span> has a transcript of an interview with @eojthebrave at DrupalCon which is surely better than this :) (at least, once we "skillify" it)</p> <p>Compare/contrast "nothing" and "new rule," including how to go about writing an eval from scratch in the first place.</p> <h3>Scenario 2: Updates to an existing rule</h3> <p><span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/ai_best_practices/-/work_items/3581832" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/ai_best_practices/-/work_items/3581832</a></span> brought in <span class="drupalorg-gitlab-issue-link drupalorg-gitlab-link-wrapper"><a href="https://git.drupalcode.org/project/ai_best_practices/-/work_items/3581672" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/ai_best_practices/-/work_items/3581672</a></span>, at least in part. I would like Claude to take another stab at it and then use the new evals framework in practice to see how the new version holds up.</p> <p>Compare/contrast "old rule" and "new rule," including how to make amendments to evals.</p> <h3>Scenario 3: The agents keep messing something up and we want to add a new eval to catch it</h3> <p>For example, "stop writing Drupal 7 code" :P </p> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <p>Have Claude do this, and also write its own docs to help.</p>