>>> [!note] Migrated issue <!-- Drupal.org comment --> <!-- Migrated from issue #3574641. --> Reported by: [littlepixiez](https://www.drupal.org/user/2568718) Related to !2 >>> <h3 id="summary-problem-motivation">Problem/Motivation</h3> <p>We need GitLab CI to clearly and consistently validate the scenario coverage we rely on for AI module QA.</p> <p>Right now we need to ensure both:</p> <ul> <li>baseline template/version smoke checks, and</li> <li>extended scenario coverage from the full scenario set</li> </ul> <p>are executed in pipeline, with expected outcomes documented.</p> <h4 id="summary-steps-reproduce">Steps to reproduce</h4> <ol> <li>Run the GitLab pipeline.</li> <li>Observe current scenario jobs and verify whether both scenario files are executed: <ul> <li><code>tests/scenarios-template-version.json</code></li> <li><code>tests/scenarios.json</code></li> </ul> </li> <li>Confirm expected results are documented and visible from logs/artifacts.</li> </ol> <h3 id="summary-proposed-resolution">Proposed resolution</h3> <p>Update <code>.gitlab-ci.yml</code> to run and document two scenario layers:</p> <p><strong>1) Smoke layer (fast matrix):</strong></p> <ul> <li>Run <code>tests/run-scenarios.sh</code> against <code>tests/scenarios-template-version.json</code> (matrix indexes 0-5).</li> <li>Purpose: quick validation of core template/version combinations.</li> </ul> <p><strong>2) Extended layer (full matrix):</strong></p> <ul> <li>Add a job to run <code>bash tests/run-scenarios.sh tests/scenarios.json</code>.</li> <li>Purpose: validate broader scenario behavior, including module resolution expectations (success/failure, skipped modules, forced dependency paths, etc.).</li> </ul> <p><strong>3) Document expected outcomes:</strong></p> <ul> <li>What each scenario set is intended to validate.</li> <li>Which assertions are expected to pass/fail.</li> <li>Where to find logs/artifacts for troubleshooting.</li> </ul> <p><strong>Scope note:</strong> this is baseline scenario validation and documentation. It is not full production image-build verification.</p> <h3 id="summary-remaining-tasks">Remaining tasks</h3> <ul> <li>[x] Create smoke matrix job for <code>tests/scenarios-template-version.json</code>.</li> <li>[x] Add GitLab CI job for <code>tests/scenarios.json</code>.</li> <li>[x] Ensure both jobs retain logs/artifacts on failure.</li> <li>[x] Document in repo docs what each scenario file validates and expected outcomes.</li> <li>[x] Validate pipeline success and expected failures across both scenario sets.</li> </ul> <h3 id="summary-ui-changes">User interface changes</h3> <p>None.</p> <h3 id="summary-api-changes">API changes</h3> <p>None.</p> <h3 id="summary-data-model-changes">Data model changes</h3> <p>None.</p>