feat: #3615987 Extend the Varbase functional testing suite to every shipped section
Issue: https://www.drupal.org/project/horizonaid/issues/3615987
Follows the shape of the Varbase Project 11.0.x and Educare 1.0.x suites: one numbered feature folder per concern, one CI job per folder, and no custom step definitions (every phrasing resolves from the Varbase E2E core steps).
What this adds
The suite grows from 3 folders to 11, covering every section the template ships:
| Folder | Proves |
|---|---|
01-website-base-requirements |
Every shipped Canvas page renders; the navigation and footer reach every section; the header collapses at a phone width and expands on a desktop. |
02-countries |
The Countries listing, a country page and its sections, the related-countries block. |
03-programs |
The Programs listing, a programme page, and the world map's country journey. |
04-resources |
The Resources listing, an article, the keyword and Program filters, the pager. |
05-events |
The Events listing, an event page, its filters and paging. |
06-impact-and-donate |
The Our Impact figures and the Donate page. |
07-quality |
Accessibility and front-end performance of the shipped pages. |
08-seo |
The metatags each page type emits, the sitemap and robots.txt. |
09-editorial |
A webmaster reaches the editorial surfaces and can create, verify and delete each content type, leaving the site as it found it. |
10-drupal-canvas |
Every shipped Canvas page opens in the Canvas editor, not just in the front end. One file per page group, because each editor mount is a heavy React SPA. |
11-permissions |
Anonymous visitors are refused every administrative surface; each role reaches only what it should. |
The old 02-editorial and 03-permissions folders are removed: their scenarios are carried
over and extended in 09-07-editorial-access.feature and 11-01-anonymous-boundaries.feature.
A written tag taxonomy
tests/TAGS.md records the taxonomy so @smoke means the same here as on every other
Varbase suite: purpose (@smoke / @regression / @acceptance / @exploratory), severity
(@critical), speed, technology, environment, status and domain tags. The canonical filters
are frozen into package.json scripts (test:ci, test:smoke, test:hotfix,
test:lane:fast, test:production, test:nightly) so nobody types an expression by hand.
Speed tags are derived from measured durations in real run reports, taking each scenario's
best observed time, not from estimates. TAGS.md also records where the taxonomy does not
fit cleanly, rather than smoothing it over: a Scenario Outline cannot carry a speed tag per
row, and the smoke lane has no Impact or Donate coverage because every scenario there
exceeds five seconds in image-derivative generation.
@javascript is deliberately unused. In varbase-e2e it is a runtime switch that fails a
scenario on any JavaScript console error, and drimage_improved creates image styles on
demand, so a freshly installed site emits harmless 404s on first paint at any width nobody
has visited yet. tests/README.md documents that behaviour, and no scenario asserts zero
console errors or that every asset returns 200.
CI
The test stage matrix goes from 3 buckets to 16: one per feature folder, plus one per page
group in 10-drupal-canvas. Each job builds its own Varbase 11 site, installs the recipe
with drush site:install recipes/horizonaid, serves it and runs only its own folder.
Defects this work found
Two are already fixed, by #3616336, which this suite's 07-quality job asserts:
the Donate page had no first-level heading, and the Our Impact figures failed contrast at
1.56:1. The following are reported but not addressed here:
- On
/resourcesthe exposed filter form's action is the Blog feed path, so Reset submits to/blog/feed?...&reset=Resetand the visitor lands on a 404. The Events listing's Reset works.04-03covers recovery by navigation instead and says why. - Every demo node shares one
createdtimestamp, and the listings sort onsticky, createdwith no tiebreaker, so rows can shuffle between requests and a visitor can miss items on a paginated listing. Nothing in the suite depends on item position for this reason. /countriesreports a serious axenested-interactiveon the map canvas, and the pager markup reports a seriouslistviolation that comes from upstream Varbase.- Canvas pages emit no
og:title,og:descriptionorog:image, so a shared Donate link previews with no headline or image.
Verification
Locally, against a DDEV build of this branch's recipe, with gitlab-ci-local:
📦 Composer validate,📐 YAML lint(484 files),🔤 Cspell,🟨 ESLintand🧩 Cucumber dry-runall pass. The dry-run resolves every step: no undefined, no ambiguous.- Folder runs:
02-countries35/35,03-programs22/22,04-resources21/21,05-events23/23,06-impact-and-donate23/23,08-seo38/38, each confirmed without relying on a retry. - The
01-website-base-requirementsand07-qualityjobs are being re-run ingitlab-ci-localafter the warm-up removal and #3616336; the pipeline here is the record that matters.
A suite-wide warm-up feature was written and then removed: the
warm up "<path>" at all testing breakpoints step visits seven breakpoints inside one step,
which exceeds the 45s per-step ceiling varbase-e2e enforces, so on the heaviest pages it
failed as a timeout and warmed nothing. The two places that genuinely need a warmed
derivative warm their own page.
Checkpoints
- File an issue
- Addition/Change/Update/Fix
- Reviewed by a human
- Code review by maintainers
AI-Generated: Yes