Protocol specification
Relayed hops
What the protocol requires of a multi-replica server that carries calls between replicas.
A multi-replica server terminates a node's websocket on exactly one replica — the owning replica — while calls may originate on any of them. The call has to cross a hop the protocol does not define. How that hop is carried is the deployment's business; what the protocol requires of it is not:
- Ownership must be fenced. A replica that believes it owns a connection it has lost will answer for a node it cannot reach. Carry a per-connection epoch on every relayed message and drop traffic fenced by a newer one.
- Ordering is per direction, per call. Control and data interleave meaningfully — a credit grant that overtakes the chunk it was issued for is a bug — so a relay must preserve the order of everything it carries in one direction of one call.
- Loss must surface, not silently truncate.
chunk_seqexists for exactly this hop. An at-most-once transport can drop a chunk, and the receiver MUST fail the stream withstream_laggedrather than deliver bytes past the gap. Credit is the only bound on relay buffering, since there is no end-to-end TCP backpressure across the hop. - The 256 KiB chunk cap keeps a relayed frame inside the message-size limit of most brokers.
A deployment with no relay routes node calls on the owning replica only; a call that lands elsewhere
fails with node_offline, which is retriable.
Attacca carries this hop over NATS, with epoch fencing against the node's row.