forgo.cloud
Sign in
Repo workspace

forkjoin-ai/gnosis

Preconditioning harnesses

distributed-inference/scripts/preconditioning/README.md
forkjoin-ai/gnosis
Repo overview Open in workbench

Preconditioning harnesses

Experiment scripts for prompt-preconditioning validation. Full findings + verdict: docs/PRECONDITIONING_VALIDATION.md.

Headline: per-request word-swap is a niche (~2%); amortized prefix compression is the form that pays — ~11% recurring prefill saving, free, ships as a shorter prompt string.

Scripts

file what it does
compress-precond.mjs (the valuable one) greedily delete/shorten a static prefix; accept only when the answer holds across a query suite. GATE=exact (hard) or GATE=first (answer-head, semantic). SYS=... to override the prefix.
greedy-precond-sense.mjs per-request word-swap A/B with fluency + sense + answer gates (the niche form).
compute-table.mjs offline per-word measured-FFN-compute table over the WordNet single-token universe (resume-aware).
fluency-gate.mjs POS-preserving + re-inflection candidate generation (fluentCandidates, analyzeSurface, inflectTo). Pure, no data deps.

Engine

a0 run distributed-inference:build-fat-station
./target/release/fat-station --knot <qwen.knot> --port 8100 --role both

Probes set the FFN skip mode per-request via the X-FFN-Leakage-Mode: mask-input-low-70 header. /precond-probe returns deterministic active_blocks + generated token ids; /exit-depth is the logit-lens early-exit probe. Both in src/bin/fat-station.rs.

Data deps (large generated artifacts, kept in ~/.flux-embed/, NOT vendored)

  • qwen2.5-0.5b.knot / qwen3-1.7b.knot — the models.
  • synonyms.json — WordNet synsets (44.5k headwords; build-synonyms.mjs).
  • ctxemb.json — contextual word embeddings (35k words, 896-dim) for the sense gate; mean-centered at load.
  • vocab: apps/edge-workers/vocab-cache/qwen3-8b-temp-vocab.json (client-side tokenizer).

Gotchas

  • A 16 GB box thrashes under the 1.7b knot (/precond-probe 0.5 s → 18 s once RAM fills). Kill stray fat-stations; use the 0.5b for long grinds (~3 s/probe).
  • Don't load the 254 MB ctxemb.json in the probing process for the per-request A/B — precompute candidates offline. (compress-precond.mjs loads it once and is fine.)
  • Keep scripts/data in ~/.flux-embed/, not /tmp — /tmp is wiped.