CI: Playwright ARIA snapshot testing is highly misleading — subsets are considered valid
>>> [!note] Migrated issue
<!-- Drupal.org comment -->
<!-- Migrated from issue #3543779. -->
Reported by: [wim leers](https://www.drupal.org/user/99777)
>>>
<h3 id="overview">Overview</h3>
<p>Over at <span class="drupalorg-gitlab-issue-link project-issue-status-info project-issue-status-7"><a href="https://www.drupal.org/project/experience_builder/issues/3543770" title="Status: Closed (fixed)">#3543770: `noUi: true` SDCs do still show up in the UI</a></span>, I pushed a commit that removed 2 list items from 2 different expected ARIA snapshots.</p>
<p>But tests continued to pass!?! 🤯</p>
<p>Found the root cause:</p>
<blockquote><p>
WTF, the snapshot tests still passed?!</p>
<p>Investigated, and turns out this is because <a href="https://playwright.dev/docs/aria-snapshots#strict-matching">Playwright defaults to saying "yep, this is valid", if the expectation is a <em>subset of the actual</em>! 🤯🫠</a></p>
<p>Looks like we need <a href="https://playwright.dev/docs/aria-snapshots#strict-matching:~:text=deep%2Dequal%3A%20Matches%20if%20the%20children%20exactly%20match%20the%20specified%20list%20in%20order%2C%20including%20nested%20children"><code>/children: deep-equal</code></a> everywhere? 😅</p>
<p>Out of scope here, opening new issue for it …
</p></blockquote>
<p>This is that issue.</p>
<h3 id="proposed-resolution">Proposed resolution</h3>
<p>Update all Playwright ARIA snapshot matching tests to use strict equality, or preferably, configure this systemwide so that we can't forget to specify <code>/children deep-equal</code>.</p>
<p>(Basically: strictest by default, opt-in to loosening.)</p>
<h3 id="ui-changes">User interface changes</h3>
> Related issue: [Issue #3543770](https://www.drupal.org/node/3543770)
issue