Issue #3624570: Let the metrics job fail when the page is stale, and refresh the page it watched drift by 9,283 lines

The metrics job passed on the 1.0.0-alpha16 tag pipeline while docs/metrics.md was 86 files and 9,283 code lines out of date. Issue #3624570.

Why it was green

Three separate reasons, each sufficient on its own: the comparison ended in || true, the job carried allow_failure: true, and nothing in it asserted anything — it printed a diff and an echo asking a human to read it.

What changes

Two things kept that diff from being an assertion, and both are answered where they arise rather than worked around in the job:

  • The page counted itself. Writing it changed the tree it had just measured, so a second regeneration disagreed with the first. Measured before this change: run 1 gave 115,154 code lines, run 2 gave 115,167, run 3 agreed with run 2. mg_excluded_files() now leaves docs/metrics.md out, and one regeneration reproduces exactly.
  • cloc counts differently per version. The page now records the version that measured it, read from the cloc_version the JSON header already carries, and the job downloads exactly that release instead of whatever apt offers (1.98 on the alpha16 run, against 2.08 here — a 600-line difference in the totals).

The file walk is sorted, because 11 of the 19 plugin rows share a count with another row and arsort() is stable, so they kept whatever order the filesystem returned — not the same order on two machines.

The job then drops || true and allow_failure, and runs on a merge request touching the page or the generator as well as on a tag. A merge request touching neither is not measured, which keeps the once-per-release shape the page was given. docs/metrics.md is regenerated in the same commit, so the first tag after this is green on its merits.

Also fixed in passing: the ratio sentence now reads "an 82%" rather than "a 82%", which is what the refreshed numbers turned it into.

Verification

  • Reproduces across machines. The page was written on macOS (PHP 8.4, APFS) and the job's own recipe was then run in a Debian container (PHP 8.5, cloc 2.08 fetched by the job's own curl line): byte identical.
  • The check fails when it should. Four probes against the finished job: the page as alpha16 shipped it fails; the refreshed page with one table cell edited by hand fails, printing the diff; the refreshed page with the cloc version deleted fails at the guard; the refreshed page untouched passes.
  • phpcs --standard=Drupal,DrupalPractice clean on scripts/, phpstan level 5 clean, both files confirmed analysed by probing an error into each.

The job runs on a tag or on a merge request touching these files, so this merge request's own pipeline exercises it — deliberately, since it is the only pipeline that can before the next tag.

Merge request reports

Loading