>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3588800. --> Reported by: [ajv009](https://www.drupal.org/user/3653917) >>> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>QA comparison quality is entirely determined by the stability of the reference baseline. When the source site is re-crawled for comparison screenshots on every ralph-loop iteration, two failure modes occur:</p> <ul> <li>The site may have changed between the original migration and the comparison, producing false mismatches.</li> <li>The comparison screenshot is taken with session state (cookie consent already dismissed, scroll position set) that differs from the original capture, producing subtle differences.</li> </ul> <p>A committed screenshot is immune to both: it is the exact image captured once, verified once, committed once &mdash; and it never changes unless the source site has genuinely changed. The recorded efi-ed-source session committed 9 PNG files (commit <code>9843d58</code>) as QA anchors for all subsequent visual fidelity work. The 20+ commit QA loop was productive precisely because the reference stayed stable. Sessions that did not do this declared visual fixes "complete" only to have the user point out regressions a stable baseline would have caught &mdash; including a full layout inversion (text above vs. below image) that survived 3+ ralph-loop iterations undetected.</p> <h4 id="summary-steps-reproduce">Steps to reproduce</h4> <ol> <li>Run a migration with multi-iteration visual fidelity passes.</li> <li>Observe each pass re-crawls the source site to take fresh comparison screenshots.</li> <li>Observe: between passes, source-site dynamic content changes (events list shows different items on different days), session state varies (consent banner blocks first capture but not later ones), and the agent compares its target output against its own previous output rather than against the original source.</li> </ol> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <p><strong>Step 1 &mdash; Add commit step to <code>site-analyzer.md</code>.</strong> After all section screenshots are captured and pass size validation (&gt; 1KB per file):</p> <pre>1. Move section screenshots to docs/migration/screenshots/source-baseline/ 2. Run: git add docs/migration/screenshots/source-baseline/ &amp;&amp; git commit \ -m "chore: source site screenshot baseline &mdash; $(date +%Y-%m-%d)" 3. Capture commit SHA: git log --format=%H -1 4. Write to plan.md: source_baseline_commit: &lt;SHA&gt;</pre><p><strong>Step 2 &mdash; Update consumers.</strong> In <code>storybook-qa.md</code> and <code>visual-verifier.md</code>, add to comparison instructions: "Load <code>docs/migration/screenshots/source-baseline/&lt;section-name&gt;-&lt;viewport&gt;.png</code> as the reference image. If this file doesn't exist (site-analyzer ran without committing), stop and flag this as a blocker &mdash; do not re-crawl the source."</p> <p><strong>Step 3 &mdash; Gitignore exception.</strong> Ensure <code>docs/migration/screenshots/source-baseline/</code> is NOT gitignored even if the rest of <code>docs/migration/</code> is. Use <code>!docs/migration/screenshots/source-baseline/</code> exception or <code>git add -f</code>.</p> <p><strong>Step 4 &mdash; Phase 1 completion criterion.</strong> Add to <code>SKILL.md</code>: "Source screenshots committed to <code>docs/migration/screenshots/source-baseline/</code>. Commit SHA logged in <code>plan.md:source_baseline_commit</code>."</p> <p><strong>Why this works.</strong> A committed screenshot has three properties a freshly-crawled screenshot does not: (1) content-addressed by git commit SHA &mdash; any post-commit agent can verify it is the same image the migration was planned against; (2) readable by any agent in any future session via <code>Read</code> tool without browser automation; (3) survives compaction because it is a file path reference, not in-context image data.</p> <h3 id="summary-remaining-tasks">Remaining tasks</h3> <ul> <li>Add post-validation commit step to <code>site-analyzer.md</code>.</li> <li>Update <code>storybook-qa.md</code> "Ground truth" subsection: load committed baselines, do NOT re-crawl source.</li> <li>Update <code>visual-verifier.md</code>: source reference is always <code>docs/migration/screenshots/source-baseline/</code>.</li> <li>Add <code>!docs/migration/screenshots/source-baseline/</code> to <code>.gitignore</code> exceptions, or document <code>git add -f</code>.</li> <li>Add Phase 1 completion criterion to <code>SKILL.md</code>: baseline committed, SHA logged.</li> <li>Verify: <code>git log --oneline | head -3</code> shows the baseline commit, <code>grep source_baseline_commit docs/migration/plan.md</code> returns a SHA.</li> </ul> <h3 id="summary-ui-changes">User interface changes</h3> <p>None.</p> <h3 id="summary-api-changes">API changes</h3> <p>None.</p> <h3 id="summary-data-model-changes">Data model changes</h3> <p>New artifact directory: <code>docs/migration/screenshots/source-baseline/</code> (committed). New <code>plan.md</code> field: <code>source_baseline_commit: &lt;SHA&gt;</code>.</p> <h3 id="summary-internal-references">Internal references</h3> <ul> <li>Source markdown: <code>RESEARCH-2026-04-28/99-final/ISSUES/Sprint-8-Patterns-To-Encode/03-NEW-3-commit-anchored-screenshot-baseline.md</code> (id: NEW-3)</li> <li>Pattern definition: <code>RESEARCH-2026-04-28/07-cross-cutting/what-actually-works.md</code> &sect; P15 (commit <code>9843d58</code>, 9 PNGs, 10 subsequent commits)</li> <li>Commit context: <code>RESEARCH-2026-04-28/03-efi-ed-source/disk-state/git-history-narrative.md</code></li> <li>Related bug: <code>RESEARCH-2026-04-28/99-final/consolidated-issues.md</code> &sect; P0-2 (blank screenshots &mdash; must be fixed before committing baseline) and &sect; P1-6 (issue files reference committed baseline)</li> </ul> <h3 id="summary-open-questions">Open questions</h3> <ul> <li>Gitignore conflict: if <code>docs/migration/</code> is broadly gitignored, the <code>git add</code> step fails silently. Recommend <code>git add -f</code> or explicit <code>!</code> exception.</li> <li>Overwrite policy: what should happen if site-analyzer is re-run on an already-committed baseline? Simplest: always commit with <code>--allow-empty</code> so SHA advances even when content unchanged.</li> <li>Storage size: source screenshots for a full site (28 sections &times; 2 viewports) at Squarespace quality can be 5&ndash;20 MB total. Acceptable but worth noting.</li> </ul>