Issue #3583743: Add multi-provider comparison page with side-by-side benchmarking.

Joshua Fernandesrequested to merge
issue/ai_empathy-3583743:3583743-multi-provider-comparison-benchmarking into 1.0.x

Summary

Closes #3583743

Adds a dedicated multi-provider comparison / benchmarking interface so teams can evaluate several AI providers against the same empathy scenarios in one run and compare them side by side, instead of running separate evaluations and comparing pages manually.

Note on the route: the issue proposed /admin/config/ai/empathy/compare, but that path is already used by the Blind Comparison form (#3583745). This feature lives at /admin/config/ai/empathy/benchmark instead.

What's included

Evaluation & data

  • New benchmark_id base field on AiEmpathyResult groups every result produced by a single benchmark run (update hook ai_empathy_update_10002).
  • EmpathyEvaluator::evaluateScenario() accepts an optional $benchmark_id; new createBenchmarkBatch() runs every selected provider against every selected scenario for N runs in a single Batch API process (parallel evaluation).
  • New BenchmarkService (ai_empathy.benchmark) aggregates a run into per-provider metric averages, provider rankings, a per-scenario breakdown, historical trends across runs, and CSV output.

Interface (AiEmpathyBenchmarkForm + AiEmpathyBenchmarkController)

  • Multi-select of 2+ providers/models (validated) and scenario checkboxes grouped by category, with configurable runs per scenario.
  • Comparison view: grouped bar chart of all four metrics across providers, radar-chart overlays per provider, provider rankings by empathy performance, and per-scenario data tables.
  • Historical trends chart across past benchmark runs for model-version regression detection.
  • CSV export endpoint and a print / save-as-PDF view for shareable reports.
  • New Benchmark menu + local-task tab.

Acceptance criteria

  •  Comparison form (multi-provider select, category scenario checkboxes, run counts)
  •  Controller with Chart.js (grouped bar, radar, per-scenario tables, rankings)
  •  Routing (/benchmark, /benchmark/{benchmark_id}, /benchmark/{benchmark_id}/export)
  •  Batch processing for multi-provider evaluation
  •  Historical trends visualisation
  •  CSV export (+ print/PDF)
  •  Automated tests
  •  Multi-provider validation

Testing

  1. Enable ai + ai_empathy, configure at least two chat providers/models, and add/confirm some scenarios.
  2. Go to Admin → Config → AI → AI Empathy → Benchmark.
  3. Select 2+ providers, pick scenarios, set runs, and Run Benchmark.
  4. On completion you're redirected to the comparison view — verify the grouped bar chart, radar overlays, rankings, and per-scenario tables.
  5. Use Export CSV and Print / Save as PDF.
  6. Run a second benchmark and confirm Historical trends renders on the landing page.

Automated tests

  • tests/src/Kernel/BenchmarkServiceTest.php — aggregation, rankings, trends, CSV.
  • tests/src/Functional/EmpathyBenchmarkTest.php — route access control, results rendering, CSV export.

Merge request reports