Write several process variables at once, recorded as one audit trail event

Closes #3628468.

What changes

  • setVariables() on ProcessControlInterface / WorkflowEngine / VariableResolver writes a name-keyed map in one scope. setVariable() is a batch of one; setVariableIfNewer() is replaced by setVariablesIfNewer(), which still compares each variable's own write time and returns the names it wrote.
  • Three queries per batch, whatever its size: one SELECT for the rows the names already have, one multi-row INSERT for the new ones, one UPDATE (CASE on the id) for the rest, both writes in one transaction; a batch with nothing to write (every value stale) returns before touching the database or the caches. Then the storage's static cache of the updated rows is reset and the orchestra_variable_list (and per-row) cache tags are invalidated once. The rows are written around the entity save, so the orchestra_variable entity save hooks no longer fire (no in-tree implementer; deletes still go through the storage).
  • One query per read: getInstanceVariables() (the engine's most frequent read) and readInstanceValue() select name, value straight from the table instead of an entity query plus a load of every row. Measured on 40 variables without Xdebug: a cold read 1.1 ms → 0.063 ms, a repeated read in one request 0.12 ms → 0.06 ms, 2 queries → 1.
  • No variable UUID: the uuid entity key is gone. Nothing reads it (variables are never referenced from outside their instance, exported or linked), and it cost a column and a unique index on the busiest table plus a UUID per insert.
  • One audit event per batch: channel renamed orchestra.variables, resource orchestra:variables/<instance>, context variables (names) + scope (+ node) in the permanent bucket, values transient and opt-in, each value capped on its own. orchestra_audit_trail follows: chain channel list, events.variables.set, schema, settings form, README, docs/audit.md.
  • The write as a diff: the batch's SELECT already reads what each existing variable held, so recordVariables() also carries those previous values. With "Also record the values" on, the subscriber hands Audit Trail the write's before and after, which it folds into the same snapshot delta as an entity update (state, delta.new, delta.original). No extra query.
  • WatchedVariablesChangedEvent replaces WatchedVariableChangedEvent: one event per batch with every watched instance-wide variable whose value changed. NoticeQueueSubscriber queues one notice for it (the body already carried a map); ParentVariableWriter writes the batch to a local parent as one write, and a remote child's notice through setVariablesIfNewer().
  • The engine's multi-variable writes use the batch: start seeding, signal / resume payloads, incident resume, the variables node, a remote subprocess's result, the child status pair.
  • docs/variables.md gains "Writing variables from code".

Cost

A 20-variable batch: 3 queries on orchestra_variable (SQLite's driver alone splits the INSERT into one statement per row) instead of ~60 (lookup, load, save per variable), 1 audit event instead of 20, 1 watched-variables event instead of up to 20.

Deploy (no update hook before beta)

After deploying, by hand:

  • Every site running orchestra, right after the code switch and BEFORE any cache rebuild (until it runs, every new variable write fails, since the old uuid column is NOT NULL with no default and a unique index, and a cache rebuild on a site with Views fails, since Views maps the installed uuid key to a field the code no longer defines):
    $manager = \Drupal::entityDefinitionUpdateManager();
    $manager->uninstallFieldStorageDefinition(\Drupal::service('entity.last_installed_schema.repository')->getLastInstalledFieldStorageDefinitions('orchestra_variable')['uuid']);
    $manager->updateEntityType(\Drupal::entityTypeManager()->getDefinition('orchestra_variable'));
    then drush cr; getChangeSummary() is empty afterwards. The installed field has to come from the last-installed schema: getFieldStorageDefinition() returns the code's definition, which has no UUID any more.
  • A site that has orchestra_audit_trail: in audit_trail.chain.orchestra, the channels entry orchestra.variable → orchestra.variables; in orchestra_audit_trail.settings, the key events.variable → events.variables. Skipping this is not neutral: a missing toggle key counts as enabled, so a site that had variable recording off would start recording every variable write. Rows already in the chain keep orchestra.variable.

Tests

  • New VariableBatchWriteTest: statement counts per kind for a 3- and a 21-variable batch, the value read back through an already-loaded entity, the list cache tag invalidated, watched variables announced once together (unchanged and unwatched ones left out), setVariablesIfNewer() skipping only the newer row. Each assertion was checked against a mutant (no static-cache reset, no tag invalidation, unchanged values announced, newer check removed, one event per variable): each mutant fails its test.
  • VariableBatchWriteTest::testTheVariablesAreReadInOneQuery: a cold read is one query on the variable table (fails against the entity query + load).
  • AuditEventsTest: a start with two variables records exactly one orchestra.variables:set event carrying both.
  • AuditTrailChainIntegrityTest::testVariablesWriteIsRecordedAsDiff: through a real chain, the stored transient bucket holds state, delta.new = ['note'], delta.original = ['decision' => 'pending'] (fails when before is not handed over). VariableBatchWriteTest asserts the previous values of existing variables only.
  • ChildVariablesTest::testOneChildBatchReachesTheParentAsOneWrite and NoticeDeliveryTest::testTheCallerIsToldOneWriteInOneNotice: a child's batch reaches a local parent as one write (one audit event) and a calling site as one notice; each fails against a one-per-variable mutant.
  • A batch where every value is older than its variable writes nothing and invalidates no cache tag (asserted in VariableBatchWriteTest).
  • Updated: OrchestraAuditTrailSubscriberTest (names permanent, values per-value capped/dropped), AuditTrailChainInstallTest, NoticeDeliveryTest.
  • Run locally green: the above plus ChildVariables, ChildStatus, Subprocess, RemoteSubprocess, EngineIncident, InitialVariables, InstanceStartedEvent, OrchestraNotificationQueue, ProcessTokens, SignalIdempotency, VariablesTask, JoinMerge, TokenVariable, AuditTrailChainIntegrity and every test that counts queries.
Edited by Frank Mably

Merge request reports

Loading
Loading