>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3588780. --> Reported by: [ajv009](https://www.drupal.org/user/3653917) >>> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>There is no systematic mechanism to prove that a component fix actually fixed the problem versus introduced a regression. The current <code>component-fixer</code> reports "fixed" based on the updated screenshot alone, with no before/after comparison or quantification of how much the component changed. This means:</p> <ul> <li>Regressions go undetected until the user manually reviews the live site.</li> <li>Over-corrections (too many changes at once) are indistinguishable from targeted fixes.</li> <li>The agent cannot prove that small iterative changes are making progress.</li> </ul> <p>The user's verbatim request is explicit: <em>"component iteration should be build with a revisions system or something not complete git tracking but something as simple as: COMPARE: Old/previous, current &amp; new component to properly break down the differences and see if the change actually fixed it, we need to have a threshold or something on the variance of the change and the code changes applied."</em></p> <p>This is the systematic long-term solution that 3-01 (before-fix screenshots) and 3-04 (one-component-at-a-time discipline) both build toward. The <code>visual-comparison</code> skill already exists and can be extended to support three-way comparison. A VideoCard text-above/text-below structural error that survived multiple ralph-loop iterations would have been caught immediately by a revision system with a structural delta check.</p> <h4 id="summary-steps-reproduce">Steps to reproduce</h4> <ol> <li>Run a <code>component-fixer</code> against a known issue.</li> <li>Inspect the component's migration directory (<code>docs/migration/components/&lt;name&gt;/</code>).</li> <li>Observe: no <code>revisions/</code> directory exists; no log of pre-fix vs post-fix state; no quantification of change direction.</li> <li>Re-run the fixer. Observe: the agent has no record of how many fix iterations have been applied or whether prior iterations converged on the source.</li> </ol> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <p>Design a lightweight, per-component artifact convention.</p> <p><strong>Directory structure:</strong></p> <pre>docs/migration/components/&lt;name&gt;/ revisions/ source.png &lt;- source site screenshot (written by storybook-qa at issue time) 1-pre.png &lt;- built state before fix #1 (written by component-fixer before edit) 1-post.png &lt;- built state after fix #1 (written by component-fixer after upload+verify) 2-pre.png &lt;- built state before fix #2 2-post.png &lt;- built state after fix #2 revisions.jsonl &lt;- machine-readable log of each revision</pre><p><strong><code>revisions.jsonl</code> format (one line per revision):</strong></p> <pre>{"revision": 1, "timestamp": "2026-04-28T14:23:00Z", "pre_screenshot": "1-pre.png", "post_screenshot": "1-post.png", "source_screenshot": "source.png", "issue_id": "issue-006", "direction": "converging", "css_delta_notes": "font-weight corrected from 400 to 600; line-height 1.2 -&gt; 1.62", "agent_assessment": "Fix narrowed the visual delta by approximately 40%"}</pre><p><strong><code>component-fixer</code> protocol additions:</strong></p> <ol> <li>Before any code change: take screenshot &rarr; write to <code>revisions/&lt;N&gt;-pre.png</code>; read <code>revisions.jsonl</code> to know the current revision number N.</li> <li>Apply code change.</li> <li>Run <code>canvas:upload</code>.</li> <li>Navigate to <code>&lt;site&gt;.cms.acquia.site?nocache=&lt;rand&gt;</code> and take screenshot &rarr; write to <code>revisions/&lt;N&gt;-post.png</code>.</li> <li>Compare: is post-fix more similar to <code>source.png</code> than <code>&lt;N&gt;-pre.png</code> was? <ul> <li>If yes: log <code>direction: "converging"</code> to <code>revisions.jsonl</code>, continue.</li> <li>If no: log <code>direction: "diverging"</code>, flag as REGRESSION, halt and surface to user before proceeding.</li> </ul> </li> </ol> <p><strong>Threshold guidance</strong> (from the user's "threshold or something on the variance" note):</p> <ul> <li>Too-small change (&lt; 1% visual difference between <code>&lt;N&gt;-pre</code> and <code>&lt;N&gt;-post</code>): flag as <em>"change too minor &mdash; may not have fixed the issue. Verify on page before declaring done."</em></li> <li>Too-large change (&gt; 50% visual difference between <code>&lt;N&gt;-pre</code> and <code>&lt;N&gt;-post</code> AND <code>&lt;N&gt;-post</code> not closer to source): flag as <em>"over-correction risk &mdash; many elements changed at once, verify each one hasn't regressed."</em></li> </ul> <p><strong>Extend the <code>visual-comparison</code> skill:</strong> add a <code>compare_three_way(source_path, pre_fix_path, post_fix_path)</code> operation. Returns a direction assessment: <code>converging</code> (post_fix is more similar to source than pre_fix) or <code>diverging</code> (post_fix is less similar to source than pre_fix). This does not require pixel-diff tooling &mdash; the agent can make this assessment visually if both images are read in the same turn. Quantitative pixel diff (e.g., <code>pixelmatch</code>) can be added as a script in iteration 2.</p> <h3 id="summary-remaining-tasks">Remaining tasks</h3> <ul> <li>Add mandatory pre-fix capture and post-fix capture to <code>.claude/agents/component-fixer.md</code>; write to <code>revisions/</code>.</li> <li>Update <code>.claude/agents/storybook-qa.md</code> so that filing an issue initializes the <code>revisions/</code> directory for the component with <code>source.png</code> and <code>1-pre.png</code>.</li> <li>Extend <code>.claude/skills/visual-comparison/SKILL.md</code> with the three-way comparison operation and direction assessment.</li> <li>Document the <code>revisions/</code> directory convention and <code>revisions.jsonl</code> format in the migrate-site skill docs.</li> <li>Add the threshold-based "too small" and "too large" flags to the fixer's post-fix evaluation.</li> <li>Verify on a fresh fix run: <code>revisions/1-pre.png</code>, <code>revisions/1-post.png</code>, and a <code>revisions.jsonl</code> entry are all created in order.</li> <li>Verify regression detection: deliberately introduce a regression and confirm the agent logs <code>direction: "diverging"</code> and halts.</li> </ul> <h3 id="summary-ui-changes">User interface changes</h3> <p>None.</p> <h3 id="summary-api-changes">API changes</h3> <p>The <code>visual-comparison</code> skill gains a <code>compare_three_way</code> operation that returns a <code>converging</code>/<code>diverging</code> direction assessment.</p> <h3 id="summary-data-model-changes">Data model changes</h3> <p>New per-component artifact convention: <code>docs/migration/components/&lt;name&gt;/revisions/</code> with <code>source.png</code>, <code>&lt;N&gt;-pre.png</code>, <code>&lt;N&gt;-post.png</code>, and <code>revisions.jsonl</code> (JSON-Lines log of each revision with <code>direction</code>, <code>css_delta_notes</code>, <code>agent_assessment</code>). The <code>revisions/</code> directory is intended as ephemeral per-session artifacts (likely <code>.gitignore</code> by default), but <code>source.png</code> should be committed as a stable reference baseline.</p> <h3 id="summary-internal-references">Internal references</h3> <ul> <li>Source markdown: <code>RESEARCH-2026-04-28/99-final/ISSUES/Sprint-3-Visual-Fidelity/05-component-revision-tracking-before-current-after-comparison.md</code> (id: P2-2)</li> <li>User note (verbatim): <code>RESEARCH-2026-04-28/00-existing-issues/unsorted-issues.md</code> (Thread 4 &mdash; revisions system)</li> <li>Deep analysis: <code>RESEARCH-2026-04-28/00-existing-issues/prior-deep-analysis-key-points.md</code> (P2 recommendation: pre-fix / post-fix capture; <code>revisions/</code> directory convention)</li> <li>Consolidated root cause: <code>RESEARCH-2026-04-28/99-final/consolidated-issues.md</code> (P2-2 entry)</li> <li>"Broke it more" feedback: <code>RESEARCH-2026-04-28/06-jamie-demo-old/95q5i-77cf5bb9-half2/user-feedback.md</code> (L10600)</li> <li>Layout-inversion evidence: <code>RESEARCH-2026-04-28/02-edu-site-mockup-ai/dbc7a979/issues-found.md</code> (CRITICAL-2)</li> <li>Working iteration reference: <code>RESEARCH-2026-04-28/03-efi-ed-source/Q3-records-5768-to-8651/visual-fidelity-iteration-pattern.md</code> (Q3 fix &rarr; upload &rarr; verify cycle, 8&ndash;15 min per iteration)</li> <li>Pattern: <code>RESEARCH-2026-04-28/07-cross-cutting/what-actually-works.md</code> &sect; P15 (Commit-Anchored QA &mdash; Screenshots as Repository Artifacts)</li> <li>Pattern: <code>RESEARCH-2026-04-28/07-cross-cutting/what-actually-works.md</code> &sect; P2 (getComputedStyle Measurement Beats Screenshot Comparison)</li> <li>Recommendation source: <code>RESEARCH-2026-04-28/99-final/migrate-site-skill-recommendations.md</code> (component-builder and storybook-qa sections)</li> </ul> <h3 id="summary-open-questions">Open questions</h3> <ul> <li>Should the pixel-diff or visual similarity calculation be automated (e.g., using a Node.js image diff library like <code>pixelmatch</code>) or agent-assessed? Given the agent can read PNGs directly, an agent visual assessment is acceptable as a first implementation; a quantitative pixel diff can be added as a script in iteration 2.</li> <li>Should revision screenshots be committed to git? The <code>visual-comparison</code> skill already established committed screenshots as QA anchors. The proposal: <code>revisions/</code> in <code>.gitignore</code> by default (ephemeral per-session artifacts) but <code>source.png</code> committed as a stable reference baseline.</li> <li>The user's note says "not complete git tracking but something as simple as." Confirms the preference for a lightweight JSONL + screenshot convention over a full git-based versioning system. Keep it simple.</li> </ul>