Deaths integration contract
The Five Deaths are implemented as standalone Rust modules — one file per death — composed into model.rs and wasm_bindings.rs by a single foreground integration step. Subagents own files; foreground owns linking.
Each death's module must match the surface declared here so:
- subagents writing the modules can work in parallel without reading each other,
- the foreground integration is mechanical (
mod x; pipeline.x = X::new(); …), - a regression that re-empties one module doesn't leak into the others.
Death #1 — src/matvec_memo.rs
pub const OP_Q: u8 = 0;
pub const OP_K: u8 = 1;
pub const OP_V: u8 = 2;
pub const OP_O: u8 = 3;
pub const OP_FFN_GATE: u8 = 4;
pub const OP_FFN_UP: u8 = 5;
pub const OP_FFN_DOWN: u8 = 6;
pub const OP_LM_HEAD: u8 = 7;
pub const OP_LM_HEAD_RANGE: u8 = 8;
pub const OP_LM_HEAD_PARTIAL: u8 = 9;
pub const OP_OUTPUT_PROJ: u8 = 10;
pub struct MatVecMemo;
impl MatVecMemo {
pub fn new() -> Self;
pub fn set_enabled(&mut self, on: bool);
pub fn set_tau_squared(&mut self, tau_sq: f32);
pub fn set_max_entries(&mut self, n: usize);
pub fn clear(&mut self);
pub fn stats(&self) -> (u64, u64, u64);
pub fn detailed_stats(&self) -> Vec<(u32, u8, u32, u32)>;
pub fn probe(
&mut self,
layer: usize,
op: u8,
rows: usize,
cols: usize,
qtype: crate::model::QType,
input: &[f32],
) -> Option<&[f32]>;
pub fn insert(
&mut self,
layer: usize,
op: u8,
rows: usize,
cols: usize,
qtype: crate::model::QType,
input: &[f32],
output: Vec<f32>,
);
}
pub fn cached_mat_vec(
memo: &mut MatVecMemo,
out: &mut [f32],
x: &[f32],
w: &[u8],
qtype: crate::model::QType,
rows: usize,
cols: usize,
n_batch: usize,
out_stride: usize,
layer: usize,
op: u8,
raw_dispatch: fn(&mut [f32], &[f32], &[u8], crate::model::QType, usize, usize, usize, usize),
);
pub fn cached_mat_vec_f32(
memo: &mut MatVecMemo,
out: &mut [f32],
x: &[f32],
w: &[f32],
rows: usize,
cols: usize,
n_batch: usize,
out_stride: usize,
layer: usize,
op: u8,
raw_dispatch: fn(&mut [f32], &[f32], &[f32], usize, usize, usize, usize),
);
pub fn cached_mat_vec_f32_with_context(
memo: &mut MatVecMemo,
out: &mut [f32],
x: &[f32],
w: &[f32],
rows: usize,
cols: usize,
n_batch: usize,
out_stride: usize,
layer: usize,
op: u8,
context_hash: u64,
raw_dispatch: fn(&mut [f32], &[f32], &[f32], usize, usize, usize, usize),
);Foreground wiring: add memo: MatVecMemo to NativeLlamaPipeline, route the Gemma/Qwen chunked and single-token layer matmuls through cached_mat_vec(&mut self.memo, ..., &mat_vec_dispatch_batch), expose memo_enable/memo_set_tau_squared/memo_set_max_entries/memo_clear/memo_stats/memo_detailed_stats in wasm_bindings.rs.
Death #1 hitlist
These are the known matmul surfaces not yet first-class under the memo / resident / durable-cache boundary. Keep this queue ordered by expected Monster impact and only mark an item closed after a focused regression test proves a warm identical input skips or hydrates.
Closed in this pass:
| Surface | Closure |
|---|---|
| LM head full/range/partial | lm_head_only, lm_head_range, and lm_head_partial now route through cached_mat_vec with distinct op ids and a shape-aware key. model::tests::lm_head_full_range_and_partial_use_matvec_memo proves full/range/partial warm hits and prevents full-vocab/range/partial bucket collision. |
| Durable matvec output cache seam | @a0n/knotgraph/layer-cache now exports R2MatVecOutputCache, an exact-key R2 helper for compact F32 matvec outputs. getOrCompute hydrates from R2, computes on miss, and writes through best-effort so storage failures do not fail inference. |
| Worker-side durable matvec hydration | optimizedTiledMatVec now has an optional durable-cache hook. LayerLiquidDO installs the KnotGraph/R2 cache when EDGE_MATVEC_OUTPUT_R2_CACHE=1, using the R2 prefix as model identity; known-qtype exact hits return before any KV/R2 weight read. |
| Phi-3 decode/lm-head matvec coverage | Phi3Pipeline now owns a resident MatVecMemo; fused QKV slices, attn output, fused gate/up halves, ffn down, and lm_head route through cached_mat_vec while preserving the existing fused tensor slicing. |
Native gnosis-uring FFI matvec admission |
Native Q4K/Q6K/F32 exported matvec kernels now have an exact resident tier keyed by qtype, dimensions, weight pointer, and input fingerprint, with FFI stats/clear/config controls for benchmark attribution. |
| Whisper decoder F32 projections | WhisperDecoder now enables an exact resident F32 memo for cross K/V prep, self/cross projections, MLP projections, and tied lm_head. WASM/NAPI wrappers expose memo enable/stats/clear for benchmark attribution. |
| Durable matvec output cache rollout default | Mesh and all 70B node Worker configs now default EDGE_MATVEC_OUTPUT_R2_CACHE=1 with the shared matvec-output-cache/v1 prefix, so distributed-inference mesh lanes hydrate exact matvec outputs from R2 by default. |
| TTS acoustic F32 projections | cached_mat_vec_f32 admits already-materialized fp32/fp16-dequant weights into the exact resident memo. TtsAcousticModel now routes mel projection plus encoder Q/K/V/O/fc1/fc2 through it and exposes memo enable/stats/clear. |
| Speech adapter F32 projections | model_speech_adapter is now part of the crate module map and routes its Q-Former K/V/Q/O/FFN/final projections through cached_mat_vec_f32 while preserving the existing forward(&self, ...) API via interior memo state. forward_reuses_resident_projection_memo_on_identical_input proves warm exact hits. |
| Vocoder decoded-weight residency | HiFiGANVocoder now keeps a resident decoded f32 tensor cache for immutable conv weights/biases, avoiding repeated backend fetch and fp16/fp32 materialization across forwards. fetch_f32_reuses_resident_decoded_tensor_cache proves warm tensor reuse and explicit cache clear. |
| Phi-3 attention-score prefix memo | cached_mat_vec_f32_with_context adds a caller-supplied discriminator for dynamic F32 "weights"; Phi-3 per-head attention now hashes the live K-prefix and memoizes raw q @ K scores before SWA mask/scale. context_hash_keeps_f32_attention_prefixes_distinct prevents wrong hits when the prefix changes, and existing Phi-3 attention tests pass through the memo-enabled implementation. |
| Llama/Gemma attention-score prefix memo | NativeLlamaPipeline now owns exact per-layer/per-head attention-score memos, enabled by the same memo_enable(true) hotpath switch. All four model.rs attention-score sites hash the live K-prefix and memoize raw q @ K scores before scale/softmax, so Rayon head loops do not contend on the main matvec memo. forward_one_uses_attention_score_prefix_memo proves warm identical decode hits. |
| Q5 gate/up pair unmasked path | The Q5_0 paired gate/up fast path now routes its unmasked branch through two existing cached_mat_vec admissions with distinct OP_FFN_GATE / OP_FFN_UP ids. q5_gate_up_pair_uses_existing_matvec_memo_contract proves the pair can warm-hit without a new compact-output layout. |
| Single-output masked leakage kernels | Q5_0 masked gate/up plus Q4K/Q6K masked down branches now route through cached_masked_mat_vec, which adds a mask-byte context hash and mask-role discriminator to the existing memo key. masked_matvec_memo_key_includes_mask_bytes proves changed masks miss and same masks warm-hit. |
| Paired output-masked Q5 gate/up | The Q5 paired output-masked branch now uses cached_q5_pair_output_masked, which keys both gate/up outputs by the output-block mask and block length before admitting distinct OP_FFN_GATE / OP_FFN_UP compact outputs. q5_pair_output_masked_memo_key_includes_output_mask_and_block_len proves changed output-mask geometry misses and same geometry warm-hits. |
| Masked memo benchmark attribution | bench-matvec now reports raw masked kernels alongside exact warm memo paths. On the synthetic Qwen-shaped release run (--iters 50 --vocab-rows 8192), warm memo measured Q5 masked gate at 1.94 us, Q5 output-masked paired gate/up at 8.34 us, Q4K masked down at 5.14 us, and Q6K masked down at 5.12 us per call, versus raw masked paths in the 0.58-10.34 ms range under local Rayon scheduling noise. |
| Priority | Surface | Current state | Next bite |
|---|---|---|---|
| P2 | Whisper encoder | Encoder batch matvecs use direct mat_vec_f32_batch; decoder now has resident F32 memo coverage. |
Decide whether encoder batch memo pays off versus the cost of hashing every 1500-frame batch row. |
| P2 | Remaining Pneuma conv output surfaces | TTS acoustic, speech adapter F32 projections, and vocoder decoded-weight residency are covered. Flow/vocoder output tensors are convolutional, not simple matvec dispatch shapes. | Define a separate conv-output cache contract before wiring; do not force them through the matvec key layout. |
| P3 | Monster masked memo shootout | Masked FFN leakage kernels are wired and synthetic bench-matvec attribution is in place. |
Run an end-to-end Monster profile with leakage modes enabled to measure real hit rates and cache residency pressure. |
Death #2 — src/amplituhedron.rs
pub struct AmplituhedronCache;
impl AmplituhedronCache {
pub fn new() -> Self;
pub fn set_enabled(&mut self, on: bool);
pub fn set_max_volumes(&mut self, n: usize);
pub fn clear(&mut self);
pub fn stats(&self) -> (u64, u64, u64);
pub fn capture(
&mut self,
prefix_hash: u64,
prefix_len: u32,
layer_lo: u16,
layer_hi: u16,
tail_residual: Vec<f32>,
kv_slab: Vec<f32>,
);
pub fn replay(
&mut self,
prefix_hash: u64,
prefix_len: u32,
layer_lo: u16,
layer_hi: u16,
) -> Option<(Vec<f32>, Vec<f32>)>;
}Foreground wiring: add amplituhedron: AmplituhedronCache field to NativeLlamaPipeline. Wasm-bindgen accessors amplituhedron_enable / set_max_volumes / clear / stats / capture / replay. The capture/replay paths know the live KV layout (in model.rs); the cache itself is layout-agnostic — it just stores and returns Vec<f32>.
Death #3 — TS-only, no Rust module
See ./AEON_FLOW_TRANSPORT_PLAN.md. Surface change is in the worker handlers (apps/worker-node1/src/index.ts) and bench coordinator (scripts/bench-trisplit.ts). No wasm_bindings.rs changes.
Death #4 — src/octonion.rs (TBD)
Plan: ./open-source/gnosis/distributed-inference/OCTONION_ROUTING_PLAN.md. Surface to be defined when implementation starts; reserve the file path so the standalone-module discipline holds.
StructuralErrorgle-author rule for model.rs and wasm_bindings.rs
These two files are integration points. Only the foreground session edits them, and only after every standalone module declared above has compiled in isolation. Subagents that need integration-time changes return text patches; the foreground applies them serially.
If a future session finds either file regressed (struct missing, accessor missing), check git diff against the committed baseline and re-apply from the standalone modules. Don't assume in-flight subagents will preserve them.