test: #3633124 Add functional tests for Varbase AI Safety

Issue: https://www.drupal.org/i/3633124

Adds the Varbase functional testing suite (@vardot/varbase-e2e) and a varbase-e2e-test job: a fresh Varbase 11 site, the recipe applied twice, 12 scenarios, no AI key.

  • CI checks the apply stops without a default model, then uses the AI module test provider (EchoAI) and installs ImageMagick.
  • 1 @ai-provider scenario(s) run in their own job only when OPENAI_API_KEY is set.

Scenarios (13):

01-01-ai-safety-stack.feature

  • Check the prompt safety and PII guardrail sets are there
  • Check the guardrails cover script injection, medical advice and email addresses
  • Check AI requests are logged and kept for 90 days
  • Check the AI log and the observability settings open
  • Check the image safety rule is on the Image media type

01-02-ai-safety-image-upload.feature

  • Verify an image the vision model does not approve is refused with the safety message
  • Verify a safe image is accepted (@ai-provider)

01-03-ai-safety-log.feature

  • Check the image check from the upload was written to the AI log

01-04-ai-safety-access.feature

  • Check the "Content editor" user cannot open the guardrails or the AI log
  • Check the "Content admin" user cannot open the guardrails or the AI log
  • Check the "SEO admin" user cannot open the guardrails or the AI log
  • Check the "Site admin" user cannot open the guardrails or the AI log
  • Check an anonymous visitor cannot read the AI log

Green locally with gitlab-ci-local: 12 passed.

AI-Generated: Yes

Checkpoints:

  • File an issue
  • Addition/Change/Update/Fix
  • Testing to ensure no regression
  • Automated unit testing coverage
  • Automated functional testing coverage
  • UX/UI designer responsibilities
  • Readability
  • Accessibility
  • Performance
  • Security
  • Developer Documentation
  • User Guide Documentation
  • Reviewed by human
  • Code review by maintainers
  • Full testing and approval
  • Credit contributors
  • Review with the product owner
  • Release notes snippet
  • Release

🤖 Generated with Claude Code

Edited by Rajab Natshah

Merge request reports

Loading
Loading