ToranaNegative result · 26 July 2026
I built a context compactor. I couldn’t establish reliable savings.
Many coding-agent workflows resend visible conversation history on successive requests. A large tool result can remain in that history long after its useful moment, so replacing it with a smaller representation looks like an obvious saving. Provider-managed history and native compaction are different cases.
I built that transformation as a Torana plugin. Then I measured it.
The result
The 21 July 2026 experiment used OMP with DeepSeek V4 Pro: five tasks, five repetitions, and three randomized arms per repetition, for 75 sessions. Costs were calculated from provider-reported usage and the historical prices used for that run.
The deterministic arm removed 4.81 MB from repeated history. Its paired median cost reduction was $0.000524 per session, with a bootstrap 95% interval of −$0.003714 to +$0.003725. Mean savings were slightly negative. The result did not establish reliable savings.
The separate model-gated arm had a paired median reduction of $0.000478, with a 95% interval of −$0.000998 to +$0.004973. It made zero paid summarizer calls; its few transformations used deterministic policy. These are not measurements of paid-summary savings.
Why the obvious model is wrong
A rewrite changes more than the removed tokens. It can invalidate a byte-identical prompt-cache prefix, it has its own summarizer cost, and it can erase information a harness expects to recover later. “Fewer tokens” and “lower bill” are not interchangeable claims.
The break-even condition
Let N be future turns, R the rewritten cached span, W the provider’s write/input rate, C its cache-read rate, D tokens removed per future turn, and S the summarizer cost. Compaction needs:
N > (R × max(0, W − C) + S) / (D × C)This simplified model assumes positive D × C and matching units: rates per token when S is in dollars. It does not model every provider’s caching rules or changes in agent behavior. Missing economic inputs or non-positive expected net prevents economically gated compaction.
What happened next
I stopped treating savings as the product promise. By then, the infrastructure around the experiment had become a local proxy with a shared request model and a WASM plugin runtime. I shifted the project toward that reusable layer.
Compaction is still an opt-in experiment inside Torana. It is no longer the reason the whole project exists. You can use the proxy to observe request activity, change model-facing tool definitions, or try your own plugin without enabling compaction at all.
This was one provider and workload. Completion varied across arms; safety-task outputs were manually reviewed, and the quality evaluation does not establish general non-inferiority. Historical prices are not current price guidance.