feat: #3615987 Extend the Varbase functional testing suite to every shipped section

Issue: https://www.drupal.org/project/horizonaid/issues/3615987

Follows the shape of the Varbase Project 11.0.x and Educare 1.0.x suites: one numbered feature folder per concern, one CI job per folder, and no custom step definitions (every phrasing resolves from the Varbase E2E core steps).

What this adds

The suite grows from 3 folders to 11, covering every section the template ships:

Folder Proves
01-website-base-requirements Every shipped Canvas page renders; the navigation and footer reach every section; the header collapses at a phone width and expands on a desktop.
02-countries The Countries listing, a country page and its sections, the related-countries block.
03-programs The Programs listing, a programme page, and the world map's country journey.
04-resources The Resources listing, an article, the keyword and Program filters, the pager.
05-events The Events listing, an event page, its filters and paging.
06-impact-and-donate The Our Impact figures and the Donate page.
07-quality Accessibility and front-end performance of the shipped pages.
08-seo The metatags each page type emits, the sitemap and robots.txt.
09-editorial A webmaster reaches the editorial surfaces and can create, verify and delete each content type, leaving the site as it found it.
10-drupal-canvas Every shipped Canvas page opens in the Canvas editor, not just in the front end. One file per page group, because each editor mount is a heavy React SPA.
11-permissions Anonymous visitors are refused every administrative surface; each role reaches only what it should.

The old 02-editorial and 03-permissions folders are removed: their scenarios are carried over and extended in 09-07-editorial-access.feature and 11-01-anonymous-boundaries.feature.

A written tag taxonomy

tests/TAGS.md records the taxonomy so @smoke means the same here as on every other Varbase suite: purpose (@smoke / @regression / @acceptance / @exploratory), severity (@critical), speed, technology, environment, status and domain tags. The canonical filters are frozen into package.json scripts (test:ci, test:smoke, test:hotfix, test:lane:fast, test:production, test:nightly) so nobody types an expression by hand.

Speed tags are derived from measured durations in real run reports, taking each scenario's best observed time, not from estimates. TAGS.md also records where the taxonomy does not fit cleanly, rather than smoothing it over: a Scenario Outline cannot carry a speed tag per row, and the smoke lane has no Impact or Donate coverage because every scenario there exceeds five seconds in image-derivative generation.

@javascript is deliberately unused. In varbase-e2e it is a runtime switch that fails a scenario on any JavaScript console error, and drimage_improved creates image styles on demand, so a freshly installed site emits harmless 404s on first paint at any width nobody has visited yet. tests/README.md documents that behaviour, and no scenario asserts zero console errors or that every asset returns 200.

CI

The test stage matrix goes from 3 buckets to 16: one per feature folder, plus one per page group in 10-drupal-canvas. Each job builds its own Varbase 11 site, installs the recipe with drush site:install recipes/horizonaid, serves it and runs only its own folder.

Defects this work found

Two are already fixed, by #3616336, which this suite's 07-quality job asserts: the Donate page had no first-level heading, and the Our Impact figures failed contrast at 1.56:1. The following are reported but not addressed here:

  • On /resources the exposed filter form's action is the Blog feed path, so Reset submits to /blog/feed?...&reset=Reset and the visitor lands on a 404. The Events listing's Reset works. 04-03 covers recovery by navigation instead and says why.
  • Every demo node shares one created timestamp, and the listings sort on sticky, created with no tiebreaker, so rows can shuffle between requests and a visitor can miss items on a paginated listing. Nothing in the suite depends on item position for this reason.
  • /countries reports a serious axe nested-interactive on the map canvas, and the pager markup reports a serious list violation that comes from upstream Varbase.
  • Canvas pages emit no og:title, og:description or og:image, so a shared Donate link previews with no headline or image.

Verification

Locally, against a DDEV build of this branch's recipe, with gitlab-ci-local:

  • 📦 Composer validate, 📐 YAML lint (484 files), 🔤 Cspell, 🟨 ESLint and 🧩 Cucumber dry-run all pass. The dry-run resolves every step: no undefined, no ambiguous.
  • Folder runs: 02-countries 35/35, 03-programs 22/22, 04-resources 21/21, 05-events 23/23, 06-impact-and-donate 23/23, 08-seo 38/38, each confirmed without relying on a retry.
  • The 01-website-base-requirements and 07-quality jobs are being re-run in gitlab-ci-local after the warm-up removal and #3616336; the pipeline here is the record that matters.

A suite-wide warm-up feature was written and then removed: the warm up "<path>" at all testing breakpoints step visits seven breakpoints inside one step, which exceeds the 45s per-step ceiling varbase-e2e enforces, so on the heaviest pages it failed as a timeout and warmed nothing. The two places that genuinely need a warmed derivative warm their own page.


Checkpoints

  • File an issue
  • Addition/Change/Update/Fix
  • Reviewed by a human
  • Code review by maintainers

AI-Generated: Yes

Edited by Rajab Natshah

Merge request reports

Loading