Introduce LLM guardrails to nudge Canvas contributors' AI use towards higher quality (#3591583) · Issues · project / canvas

Introduce LLM guardrails to nudge Canvas contributors' AI use towards higher quality

>>> [!note] Migrated issue   Reported by: [wim leers](https://www.drupal.org/user/99777) Related to !1151 >>> <h3 id="overview">Overview</h3> It's 2026. The majority of software developers is either using <del>AI</del> LLMs (but everybody says "AI" despite it not actually being intelligent so I'll run with it anyway) voluntarily or is pressured to do so. Whichever your stance may be, it is the reality. In Drupal core (see <a href="https://www.drupal.org/project/drupal/issues/3585894" title="Status: Needs review">#3585894: LLM harm reduction in Drupal core contribution, AGENTS.md guidelines</a>). But also in Drupal Canvas. Drupal CMS already adopted it (<a href="https://git.drupalcode.org/project/drupal_cms/-/work_items/3569529" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/drupal_cms/-/work_items/3569529</a>). Most of Drupal Canvas is sponsored by Acquia: it pays dozens of people to work on it. I'm one of them. Acquia definitely expects us to build Drupal Canvas faster using AI. But there are also a solid number of community contributions to Canvas. In fact, increasingly so. And as one of the main reviewers of Canvas MRs, it's been a fact that for months most of those are generated by AI. So, today's reality: sprawl: <ol> <li>High speed/throughput. (IOW: letting LLMs generate an abundance of code is easy. ) </li><li>Lots of #1 is hard to maintain. Much LLM-generated code is resulting in parallel (semi-duplicate) implementations of existing infrastructure (both in tests and actual code). It should do the necessary work: check both Drupal Canvas & core for existing infrastructure and best practices for that infrastructure, generate code with that taken into account. </li><li>So far to a lesser degree in comments/prose (<a href="https://noslopgrenade.com">https://noslopgrenade.com</a>). </li></ol> So: let's accept the reality. Just like @nod_'s core issue above, "LLM harm reduction" is a goal. For Canvas, I have no other choice but to go further than that though: "quality increase" is the main goal. References: Dries (<a href="https://dri.es/never-submit-code-you-do-not-understand">1</a>), Drupal CMS (<a href="https://git.drupalcode.org/project/drupal_cms/-/blob/2.x/AGENTS.md?ref_type=heads">1</a>), Matt (<a href="https://mglaman.dev/blog/drupalorg-cli-080-gitlab-issue-fork-and-merge-request-commands#:~:text=drupalorg%20mr%3Alogs-,Agent%20Skills,-The%20CLI%20ships">1</a>), Théodore (<a href="https://tresbien.tech/blog/proposal-llm-policy-drupal-core-contribution/">1</a>1), webchick (<a href="https://www.drupal.org/project/drupal/issues/3585894#comment-16588685">1</a>), and many more. <h3 id="proposed-resolution">Proposed resolution</h3> Get an initial set of minimal, generalist <code>AGENTS.md</code> files committed to Canvas: <a href="https://git.drupalcode.org/issue/canvas-3591583/-/blob/3591583-AGENTS.md/AGENTS.md">rendered AGENTS.md in this MR</a>. <dl> <dt>Out of scope</dt> <dd> To keep the scope of this issue manageable, these are for follow-ups: <ol> <li>modifying any existing files or docs — this MR is only adding new files </li><li>specialized agents for specific workflows or subdirectories </li><li>skills (either custom ones or existing ones) </li><li>prose guidance </li><li>provide extra guidance for known-to-be-organically-grown-and-immature areas of the codebase (internal HTTP API organization → <a href="https://git.drupalcode.org/project/canvas/-/work_items/3521041" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/canvas/-/work_items/3521041</a>, ComponentSource plugin infrastructure → <a href="https://git.drupalcode.org/project/canvas/-/work_items/3520484" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/canvas/-/work_items/3520484</a>, etc.) </li></ol> </dd> <dt>Principles</dt> <dd> <ol> <li><a href="https://en.wikipedia.org/wiki/KISS_principle">KISS</a></li> <li>Optimize for humans first, LLM agents second. (Not the <code>AGENTS.md</code> file itself, but the overall project organization.)</li> <li><code>CONTRIBUTING.md</code> is for humans, <code>AGENTS.md</code> for AI agents. Duplication between these two is okay. Both should point to more granular docs that can be used by both: LLMs should consume the same docs as humans wherever feasible.</li> <li><a href="https://docs.claude-mem.ai/progressive-disclosure#the-problem-context-pollution">Progressive disclosure to avoid context pollution.</a></li> <li>Use deterministic tools to avoid context pollution (e.g. linters).</li> <li>Strictly omit environment assumptions.</li> </ol> Most of these speak for themselves. Context for a few: <ul> <li>#2 + #3: <code>docs/intro.md</code> contains "big picture" docs by @phenaproxima. It should serve both humans and LLMs. So <code>AGENTS.md</code> should point to it.</li> <li>#3 ensures that tweaks to nudge the LLM to provide better results also improves docs for humans. This is why the existing <code>docs/*.md</code> files are reused.</li> <li>#5: use deterministic tools (PHPCS, PHPStan, eslint, stylelint, prettier …) rather than probabilistically hoping that the LLM will remember the thousand intricate code style details we'd have to describe in prose. Which would be … context pollution — see #4. </li><li>#5 bonus: this are essentially "automated reviews for baseline expectations", that also provide faster feedback to humans' MRs. Which means that once an MR passes linting, MR reviews can focus on the actual logic, organization, etc. </li><li>#6: @mglaman shared how much e.g. the <code>mcp_server</code>'s <a href="https://git.drupalcode.org/project/mcp_server/-/blob/65c454f6525e280a533ddac756b1218ec01f65af/AGENTS.md"><code>AGENTS.md</code> gets in the way</a> because it hard-codes the preferences of the lead maintainer. This is great for the lead maintainer, but is actually a barrier to contribution. </li> </ul> </dd> <dt>Dev environment</dt> <dd>Ideally, the LLM can spin up development environments as needed. However, Canvas' recommended development environment is currently in an awkward state. There are four: 1) "bare metal" (local webserver etc), 2) core's recommend-project + <a href="https://github.com/drupal-canvas/ddev-drupal-xb-dev">a DDEV add-on</a>, 3) <a href="https://github.com/balintbrews/drupal-canvas-dev">https://github.com/balintbrews/drupal-canvas-dev</a>, 4) <a href="https://git.drupalcode.org/project/canvas/-/work_items/3587161" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/canvas/-/work_items/3587161</a>. 👉 Work is happening to unify those. But it should be done in a way that doesn't violate principle #6. See <a href="https://git.drupalcode.org/project/canvas/-/work_items/3587161" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/canvas/-/work_items/3587161</a>. Canvas' CI setup uses the <code>composer run …</code> + <code>npm run …</code> commands precisely to ensure consistency between development and CI, so stick to just those commands for now. </dd> <dt>Prior work that facilitated this</dt> <dd> The following helped make this <code>AGENTS.md</code> file as lean as it is <ul> <li><a href="https://git.drupalcode.org/project/canvas/-/work_items/3591245" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/canvas/-/work_items/3591245</a></li> <li><a href="https://git.drupalcode.org/project/canvas/-/work_items/3591249" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/canvas/-/work_items/3591249</a></li> <li><a href="https://git.drupalcode.org/project/canvas/-/work_items/3585788" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/canvas/-/work_items/3585788</a></li> <li><a href="https://git.drupalcode.org/project/canvas/-/work_items/3572401" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/canvas/-/work_items/3572401</a> (and many since) → automate reviews using PHPCS ⇒ faster feedback for both humans & LLMs</li> <li><a href="https://git.drupalcode.org/project/canvas/-/work_items/3576694" class="drupalorg-gitlab-link">https://git.drupalcode.org/project/canvas/-/work_items/3576694</a> (and many since) → automate reviews using PHPStan ⇒ faster feedback for both humans & LLMs</li> </ul> </dd> </dl> <h3 id="ui-changes">User interface changes</h3> None. > Related issue: [Issue #3569529](https://www.drupal.org/node/3569529) > Related issue: [Issue #3591245](https://www.drupal.org/node/3591245) > Related issue: [Issue #3587161](https://www.drupal.org/node/3587161)

issue