forgo.cloud
Sign in
Repo workspace

forkjoin-ai/gnosis

Frame Coalescing Protocol — `FlowSendBatcher` on the Trisplit WS Hot Path

distributed-inference-host/FRAME_COALESCING_PROTOCOL.md
forkjoin-ai/gnosis
Repo overview Open in workbench

Frame Coalescing Protocol — FlowSendBatcher on the Trisplit WS Hot Path

This file is the falsifiability protocol for wiring FlowSendBatcher into the host-side station-ws.ts send path. It is the prediction ledger, not a results document. Every empirical claim below is marked as a prediction (P*) or a falsification trigger (F*); nothing here is an observation.

Companion documents:

  • TOPOLOGY_CEILING_BREAKERS.md §4 (same dir's sibling open-source/gnosis/distributed-inference/) — the design source. This protocol is the Bule-1 falsifiability gate for the lever named there.
  • BOWL_MESH_FALSIFIABILITY.md (sibling open-source/gnosis/distributed-inference/) — the canonical tone / prediction-ledger template this file mirrors.
  • ACKERMANN_CERTIFICATION_PROTOCOL.md (same sibling dir) — canonical section structure (headline reframe, what is being certified, the contract, the calibration window, predictions, triggers, honesty boundary, composition, pointers). This file inherits both shapes.
  • open-source/aeon/src/flow/WebSocketFlowTransport.ts — the wire site. Already exposes WebSocketFlowTransportOptions.coalesce and WebSocketFlowTransportOptions.batching; the trisplit station does not pass either today.
  • open-source/aeon/src/flow/FlowSendBatcher.ts — the batcher being wired in. Microtask-boundary flush, 16-frames / 64-KiB caps, non-flow passthrough.
  • apps/pneuma-think/src/trisplit-station-ws.ts — the per-layer send loop that pays the framing cost today.
  • src/bin/bench-teleportation.rs (this package) — the Death #1 matVec-memo bench whose runtime must not regress under coalescing.

1. Headline reframe

Same-turn outbound WebSocket sends from the trisplit station today each pay separate socket.write framing overhead: a TLS write per call, an event-loop tick per call, and the per-frame WS encapsulation. The FlowSendBatcher already shipped in @a0n/aeon coalesces frames queued inside the same microtask into one contiguous socket.send() call. The predicted uplift, before any compounding with Pair X, is in the 1.05× – 1.20× band on warm-WS per-layer p50. With Pair X live (TOPOLOGY_CEILING_BREAKERS.md §4 Leg 2), the predicted compounding band widens to 1.10× – 1.25× because the residual-seed dispatch and the canonical split-c tail land in the same microtask for a non-trivial fraction of layers.

This is a topology-side-of-the-wire optimization, not a kernel one. It costs nothing on workloads with sparse traffic and cannot make back-pressured workloads worse than the caps allow. The asymmetry is the load-bearing claim: the win is concentrated on a specific frame pattern (tightly-spaced, same-microtask, same-socket, single-flow-frame payloads) and is zero elsewhere.

2. What the coalescing actually optimizes

Three line-items the wire pays today, in descending order of magnitude on the trisplit hot path:

  • Per-send() TLS framing. Every socket.send() issues a TLS application-data record. The TLS record header plus MAC plus AEAD tag is paid per call regardless of payload size. Two 21-KiB sends pay two TLS records; one 42-KiB send pays one.
  • The socket.write event-loop turn. Even on a CF Worker isolate, each send() is a host-side I/O dispatch. Coalescing collapses N dispatches into one within a microtask window.
  • The per-frame WS encapsulation. The 10-byte aeon-flow header rides per inner frame regardless (the batcher concatenates inner frames, it does not strip headers); the saving here is on the outer WebSocket frame header (2–14 bytes per outer frame) plus mask-key bytes (4 bytes per client-to-server outer frame).

The wins amortize over many tightly-spaced frames. Sparse traffic — a single send, then the next-turn dispatch, then another single send — sees no gain because the batcher's microtask flush still fires after each lone enqueue. The coalescing is silent and idempotent there; it costs ≤ 1 microtask of latency for the single-frame case.

3. The flow batcher contract

The load-bearing assumption is the flush trigger semantics. Read from FlowSendBatcher.scheduleFlush (which delegates to a polyfilled queueMicrotask — Promise.resolve().then(task) on runtimes that lack the global):

  • Frames enqueued via send() schedule a single microtask if one is not already scheduled.
  • All frames queued in the same microtask flush together at end-of-tick via one call to the underlying sendNow.
  • Cross-microtask frames (a send() that lands after the microtask fires) flush independently in their own microtask.
  • Caps that force an immediate synchronous flush: payload count ≥ 16 (DEFAULT_MAX_FRAMES_PER_BATCH), aggregate bytes > 64 KiB (DEFAULT_MAX_BYTES_PER_BATCH), or a payload that is not a single flow frame (!isSingleFlowFrame) or that is itself larger than the byte cap.
  • A close() call drains the queue synchronously before tearing down the underlying sink.

If the batcher's actual flush trigger is different (timer-based, byte-threshold-only, or end-of-task rather than end-of-microtask), the prediction window in §5 needs refit. The protocol relies on microtask boundary specifically because that is the JS-runtime guarantee that "fires concurrently in the same turn" implies "ends up in the same flush."

4. Composition with Pair X (Leg 2)

Per TOPOLOGY_CEILING_BREAKERS.md §4, Leg 2 of the topology breakers dispatches the residual-seed frame for layer l+1 concurrently with the canonical split-b for layer l. The two frames target different sockets (different downstream stations), so direct coalescing between them gives no benefit.

The compounding win comes from two narrower patterns:

  • Same-peer same-microtask collapse. A small fraction of Pair X dispatches land on a peer that is also receiving a residual frame from the canonical chain in the same microtask. Those two frames share a socket and coalesce.
  • Decode-tail control frames. During decode, the trisplit station emits a stream of small control / ack / keepalive frames in tight succession to the same peer. Each of those today pays full per-frame framing overhead. Under coalescing they collapse into one socket.send() per microtask burst.

The composition is therefore additive on the per-layer wall (the big payload frames don't coalesce across sockets) and subtractive on the per-layer tail (the small control frames do). The §5 prediction bands account for both terms separately.

5. Predictions

All five are predictions, not observations. They will be falsified or upheld by re-running the warm-WS per-layer bench against the trisplit station with coalescing enabled.

  • P1. Standalone TPS uplift on warm-WS p50 is ≥ 1.05× and ≤ 1.20×. Justification: small-control-frame coalescing during decode is observable on any non-trivial layer chain; the upper bound is capped by the fact that the big payload frames do not coalesce across sockets and dominate the layer wall.

  • P2. When Pair X is live (Leg 2 in TOPOLOGY_CEILING_BREAKERS.md §4), compound TPS uplift over the pre-coalescing, pre-Pair-X baseline is ≥ 1.10× and ≤ 1.25×. Justification: the same-peer same-microtask collapse contributes a modest additional fraction on top of the standalone band, while the big-payload-no-cross-socket-coalescing cap keeps the upper bound modest. The bands overlap by design — the compound win is not guaranteed to exceed the standalone win in every workload, only on the warm decode pattern Pair X exists to exploit.

  • P3. No regression on the Death #1 matVec memoization bench at src/bin/bench-teleportation.rs. The bench drives the Rust matVec memo on a deterministic prompt; coalescing operates on the host-side WS path that is upstream of the matVec call, so end-to-end wall time should land within measurement noise of the pre-coalescing baseline (± 2% on p50 over ≥ 20 rounds).

  • P4. With GNOSIS_FRAME_COALESCE=0 set in the environment, the station's behavior is byte-identical to a tagged pre-coalescing baseline. Specifically: identical aeon-flow frames on the wire in identical order, identical TLS-record-count under tcpdump inspection on a controlled local capture, and identical per-frame timestamps within OS scheduling noise.

  • P5. Per-layer p50 wire time (currently observed in TOPOLOGY_CEILING_BREAKERS.md §2 at ~188 ms best, ~226 ms mean — the 138 ms best is split-b alone; the per-hop p50 is the relevant comparand) is reduced by 5–10 ms after coalescing wires. The band reflects that the savings are dominated by event-loop-turn collapse rather than payload-size effects.

6. Falsification triggers

These are the curves and points that would kill the wiring. Triggers, not observations.

  • F1. Standalone TPS uplift < 1.02× (within noise of zero). The premise that the trisplit hot path emits enough tightly-spaced same-microtask frames to amortize the batcher overhead would then be wrong — either the station's send pattern is too sparse, or the cost of the microtask-queued enqueue is itself non-trivial on the CF Worker runtime. Either way the lever does not exist.

  • F2. Compound uplift with Pair X is below the standalone gain. Coalescing actively hurts when Pair X is on. The most likely cause is that Pair X's concurrent dispatch already saturates the socket and the batcher's microtask delay adds latency where there was none before. Recoverable by gating coalescing off when Pair X is live, but only after the regression is characterized.

  • F3. The Death #1 bench at src/bin/bench-teleportation.rs regresses by more than 2% on p50 wall time over ≥ 20 rounds. The most concerning cause is that frame coalescing accidentally batches matVec-memo'd writes in a way that breaks the memo's hot-path assumption (e.g. the memo expects each layer's residual to land in its own write turn for cache-locality reasons). Recoverable by scoping the batcher to control frames only.

  • F4. GNOSIS_FRAME_COALESCE=0 does not produce byte-identical behavior to the tagged baseline. The flag-disable path is buggy — either the env var is not actually disabling the batcher, or the batcher's pass-through-when-disabled code path is itself producing a different wire pattern (extra empty flushes, reordered frames, etc.). This is a primitive-level failure that blocks shipping regardless of the §5 uplift, because the rollback story depends on this flag.

  • F5. Per-layer p50 wire time increases after coalescing wires. The batcher adds microtask latency without saving any framing cost — the worst case of F1, where the workload's pattern is so unfriendly to the cap-based flush that the microtask delay dominates. Coalescing must be disabled by default on this path.

Any of F1–F5 reverts the wiring to the disabled-default state. None of them by themselves invalidates the batcher or the topology plan; the batcher remains correct and load-bearing on aeon-flux and dashrelay, and Pair X (Leg 2) remains the topology lever it always was. The protocol decides only the wire-to-the-trisplit-WS question.

7. Composition with Amplituhedron coordinator (Leg 4)

The Amplituhedron prefix cache (sibling AMPLITUHEDRON_PLAN.md and the AmplituhedronFalsifiability.lean module) is orthogonal. The coordinator skips prefill entirely on warm prefixes; coalescing reduces per-frame cost on the frames that do fly. The composition is multiplicative on workloads with both a warm prefix and an in-flight decode stream: the cache reduces the count of frames; coalescing reduces the per-frame cost of the survivors. Neither lever's prediction depends on the other; the §5 bands stand whether or not the Amplituhedron cache is live in the same run.

The implementation rule from AMPLITUHEDRON_PLAN.md §5 carries over: coalescing does not run during volume capture or replay (capture freezes the source-of-truth wire pattern; replay reproduces exactly the captured bytes). Coalescing is a live-decode-path optimization only.

8. Honesty boundary

Nothing in §5 has been measured. The FlowSendBatcher contract in §3 is read from FlowSendBatcher.ts directly: scheduleFlush calls queueMicrotask (or its Promise.resolve().then polyfill), and the immediate-flush caps are the literal DEFAULT_MAX_FRAMES_PER_BATCH = 16 and DEFAULT_MAX_BYTES_PER_BATCH = 64 * 1024 constants in that file. If the wired-in version of the batcher diverges from this surface — a configuration override on the caller side, a CF Worker isolate that implements queueMicrotask as a different scheduler primitive than the spec describes, or a future patch that switches flush to timer-based — the prediction window shifts and this file needs amendment before the bench is treated as a gate.

The bench is the gate. Until it runs, the predictions are predictions. The protocol surfaces them so a later snapshot can be compared against this file rather than against a moving target.

If F1–F5 all fire, the wiring has been falsified at the same operational level the binary mask was falsified at on 2026-05-03. The previous-snapshot rule from STANDING_WAVE_STATUS_2026_05_03_AFTER_FALSIFICATION.md (sibling dir) applies: this file becomes the historical record, an addendum captures the falsification, the wiring is reverted to its disabled-default state, and the work moves to the next lever in TOPOLOGY_CEILING_BREAKERS.md.

9. Pointers

  • open-source/gnosis/distributed-inference-host/src/station-ws.ts — the wire site. The WebSocketFlowTransport instance constructed here must opt in via { coalesce: true } or an explicit batching config.
  • open-source/aeon/src/flow/FlowSendBatcher.ts — the batcher primitive. Microtask flush, 16-frame / 64-KiB caps, non-flow pass-through.
  • open-source/aeon/src/flow/WebSocketFlowTransport.ts — already threads coalesce and batching through fromSocket and connect. WebSocketFlowTransportOptions is the surface to use.
  • open-source/gnosis/distributed-inference/TOPOLOGY_CEILING_BREAKERS.md §4 — the design source. This protocol is the Bule-1 gate for that lever.
  • src/bin/bench-teleportation.rs (this package) — the composition test for P3 / F3. The Death #1 matVec memo bench must not regress.
  • apps/pneuma-think/src/trisplit-station-ws.ts — the per-layer send loop whose framing overhead this wiring targets.