Issue #3608701 by codeitwisely, marcus_johansson: Add monthly rollup and retention purge for usage log

Jérôme Tchaniarequested to merge
issue/ai_metering-3608701:3608701-retention-purge-rollup into 1.0.x

Closes https://www.drupal.org/project/ai_metering/issues/3608701

What changed

  • New ai_metering_rollup table: monthly aggregates per user, provider, model and operation, including summed latency so tokens/second and average response time stay computable after raw rows are gone (hook_update_9019 + hook_schema).
  • New RollupManager service and the module's first hook_cron: rolls every complete past month into the aggregate table (idempotent, transactional), then purges raw rows older than the retention window in bounded batches (10k rows, max 20 batches per run).
  • Structural guarantee: a month is only purgeable once its rollup completed, so purge-before-rollup data loss cannot happen. The kernel test asserts it.
  • New usage_retention_days setting, default 0 = keep raw rows forever (purging is opt-in). Exposed in the settings form under Quota defaults.
  • Dashboard: past months with a completed rollup are served from the aggregate table; the current month keeps reading the raw log. Verified live that the June dashboard renders identical numbers from both sources.
  • Dominant provider for rolled-up months = highest summed cost (the raw path uses highest single-call cost); slightly different and arguably more representative.
  • CostReporter range queries stay on the raw log: like the usage log view, they only see retained rows once purging is enabled.

Testing steps

  1. Run drush updatedb, then drush cron.
  2. Check the dashboard for a past month: totals unchanged, now served from ai_metering_rollup.
  3. Set 'Raw usage log retention (days)' to e.g. 90, run cron on a site with older data: raw rows beyond 90 days are deleted, historical dashboard totals survive.

Verified live (Drupal 11.3.12, PHP 8.4): June dashboard identical before/after the switch (68 calls, 8 editors, USD 0.3604). Suite green (136 tests, 529 assertions), PHPCS and PHPStan clean.

AI assistance

Implemented with AI assistance (Claude Code), verified end to end on a live site.

Merge request reports