Issue #3583741: Add scheduled cron-based empathy evaluation with threshold alerts

Joshua Fernandesrequested to merge
issue/ai_empathy-3583741:3583741-scheduled-cron-evaluation-alerts into 1.0.x

Closes #3583741

Adds automated, cron-driven empathy evaluation with threshold alerting, so model regressions (e.g. quality drops after a provider update) are caught without anyone manually triggering an evaluation — aligning with the Drupal AI 2026 emphasis on autonomous AI responding to schedules/triggers.

What's included

  • EmpathyScheduledEvaluator service (ai_empathy.scheduled_evaluator):
    • runIfDue() — cron entry point; runs when the configured interval has elapsed or the evaluation provider/model has changed.
    • Frequencies: every 6h / 12h / 24h / 48h / weekly; last-run tracked via StateInterface.
    • Provider-change detection — a change triggers a full run and an alert (first observation only records a baseline, so it never false-fires).
    • One run per scenario per cron cycle to keep execution bounded; per-scenario failures are caught and logged.
    • Threshold breach detection across decision accuracy / empathy / explanation quality (consistency is skipped — it needs multiple runs). Breaches and provider changes are written to the log and emailed.
  • hook_cron() → delegates to the service; hook_mail() → builds the threshold_alert email.
  • Settings → Scheduled Evaluation section: enable toggle, frequency, scenario selection (empty = all), provider/model (defaults to the evaluation model), and alert email, with #states gating.
  • Config schema + install defaults for the scheduled_* keys, plus ai_empathy_update_10003() to backfill defaults on existing installs.
  • README usage section.

Design decisions

  • One run per scenario per cycle — deliberately trades statistical depth for bounded cron execution time (as noted in the issue). Consistency isn't alerted on because it can't be measured from a single run.
  • Email-only alerts via hook_mail(), matching the issue scope (no Slack/other channels).
  • Provider/model change is treated as an alert-worthy trigger in its own right, not just a threshold check.
  • Scheduled runs go through the normal EmpathyEvaluator, so metrics contributed by other submodules (tone/governance) are scored and observed automatically.

Testing

  • Unit — frequency→interval mapping, isDue, providerChanged (incl. baseline behaviour), collectBreaches (only sub-threshold metrics flagged), and disabled → no-op (evaluator never called).
  • Kernel — service is registered, cron is a no-op when disabled (no last_run written), and the threshold_alert mail builds a subject and body.

Manual:

  1. Settings → Scheduled Evaluation: enable, pick a frequency, scenarios, provider/model, and an alert email.
  2. Run drush cron (or wait for cron). Confirm results are created, a summary appears in the log, and an alert email is sent when a metric is below threshold.
  3. Change the evaluation provider/model and run cron again → confirm a provider-change run + alert.

Merge request reports