Tags

1.0.0-beta3

1.0.0-beta3 - Portable ground truth and outside-in verdicts.

Highlights since 1.0.0-beta2:
- Row assertions accept fixture UUIDs, so shipped datasets grade any
  fresh provision without re-keying (#3594736).
- every_eval_ever envelopes import into the results table, so outside-in
  and in-process verdicts share one screen (#3594739).
- Unreadable assertion input fails in place instead of grading what is
  left (#3594738).

1.0.0-beta2

1.0.0-beta2 - Trust hardening and a documentation site.

Sprint 6 and the post-sprint trust work, plus the first published
documentation site.

Highlights since 1.0.0-beta1:

- Judge discrimination resolution: validation now estimates the smallest
  quality difference a judge can actually see, and the optimizer refuses to
  claim a gain below that floor.
- Tool assertions can name arguments and order, and the run record carries
  each call's arguments (scalars only, truncated, credential-looking names
  redacted). The judges that read the record are shown arguments too.
- An agent-mode run says so when nothing it uses can observe behavior rather
  than text, read from a new observes_actions flag on the grader attribute.
- Per-grader score scales, so a binary judge is no longer forced onto a 0-5
  gradient.
- Documentation site at https://project.pages.drupalcode.org/ai_eval, 23 pages
  built by MkDocs, plus a first ai_eval.api.php.

Full detail in CHANGELOG.md under 1.0.0-beta2.

1.0.0-beta1

1.0.0-beta1 — Trustworthy Beta (Sprint 4). Adds pluggable rubric check kinds (target_match, command, score_delta, chrF) and check-executor-provided schemas (#3594701, #3594704) on top of the errored-judge trust fix (#3594703).

1.0.0-alpha10

Release 1.0.0-alpha10

Adds Wilson confidence interval and Cochran sample-size math to
scoring/reporting, fixes three optimizer review findings (aggregated
baseline gate, propose-margin ordering, bounded LLM judge input),
exposes five previously config-only settings in the admin form, and
rewrites the Cochran sample-size warning for clarity (now fires on
any below-floor dataset, with plain-language framing and concrete
actions). Supersedes the unpublished alpha9 tag.

1.0.0-alpha8

Release 1.0.0-alpha8

Fixes form TypeError when saving EvalTarget (question_pass_threshold
/ response_char_limit / threshold typed properties rejected raw
response char limit feature and judge-validation hardening.

1.0.0-alpha7

1.0.0-alpha6

1.0.0-alpha6: Config entities, chat mode, CallerContextInterface, PHP 8.3 attributes, OOP hooks

1.0.0-alpha5

alpha5: 11 bug fixes from cross-model code review (Claude + Codex)

1.0.0-alpha4

1.0.0-alpha4: UI polish, select fields, meters, dashboard layout, docs update

1.0.0-alpha3

Remove project-specific graders, ship 5 generic graders

1.0.0-alpha2

1.0.0-alpha1