Issue #3623524: Tell the caller what its remote child is doing: signed notices, a handshake, whose runs are whose, and the reads that were missing

Implements the communication half of #3623524: two Orchestra sites agree terms, and the one running a child tells the one that asked for it what is happening.

Before this, a caller found out on its next sweep. A child finishing just after one ran stayed invisible until the next.

What is here

Agreeing terms. WireVersion is the number both ends compare, carrying a minimum as well as a current so the range between them states that both ends can still hold a conversation rather than labelling a release. Checked in both directions: the remote refuses a caller it is too new to read, and the caller refuses a remote too new for it, which that end cannot know. The exchange happens when the remote is saved and the form reports it — after the save, never instead of it, because a remote is routinely configured before the site it names exists. A refused handshake records nothing: half an agreement would read as an agreement.

Knowing whose run it is. Nothing recorded could address a notice. initiator is a user id, and over the API that names the account credentials authenticate as — one account can serve several callers. So the instance records a caller, asked for through a resolver collected exactly as tenant resolvers are. A site with no API installed has nothing tagged and records nobody, which is the honest reading of its own run.

Telling the caller. Queued rather than sent: InstanceEndedEvent's own documentation says a listener must defer outbound calls, because these fire inside the advancing worker's transaction. Sending there would announce runs that then roll back, and a hanging caller would hold the worker advancing everybody else's runs. Delivery has its own queue for the same reason, retries on the engine's attempt budget with a doubling wait, and when the budget is spent names the caller, the event and the last error — a notice dropped in silence is indistinguishable from a run that never moved.

Believing the caller. One address per remote, so an arriving notice names its sender by where it arrived rather than by anything in a body its sender chose; a secret one remote leaks cannot forge a notice from another. The timestamp is signed with the body, since a timestamp anybody can edit bounds nothing. Tolerance is symmetric, because a fast clock cannot be told from somebody holding a notice to use later. Every way of failing to be that remote is refused alike — telling them apart would say whether a handshake exists.

Capping the addresses. They are open, so refusing an unsigned post still costs the work of checking. A settings page caps how often one remote is answered and how often the addresses answer in total — the second catching somebody working through each address in turn, which the first never sees. Both ship at no limit, count nothing until request limits are on for the site, and the status report says so while they are off.

Asking twice. A start commits on the remote and then has to travel back; a caller whose connection drops cannot tell a request that never arrived from one that worked. An idempotency key makes the second ask return the run the first made, scoped to the caller as well as the key.

The secret

Minted per caller by the end that signs, handed back exactly once, readable nowhere afterwards — so losing it and revoking it are the same sentence. Kept in state on the caller's side, because configuration is exported and a secret in an export is a secret in a repository. A test asserts it appears nowhere in the exported config.

Coverage

Roughly 40 new tests across orchestra_api, orchestra_client and orchestra_server_api. Every rule proved in both directions by removing it and watching the named test fail — 30 of 31 mutations caught.

The one that cannot be: hash_equals versus === in the signature comparison differs only in timing, so no behavioural test can distinguish them. The guard stays; I would rather say that than write a flaky timing test and call it covered.

Deliberately not here

The initiator change to account-login strings. It touches 33 call sites across 13 files including ReadAccess and InstanceReadAccessCheck, and orchestra's read check short-circuits on initiator before the scope is consulted — so it is an access grant, not just an attribution field. It wants its own MR and its own audit rather than riding along at the end of this one.