Updated November 2026 · Trove v0.4.0
Measured results.
How much of a coding-agent bill Trove saves on scripted sessions replayed through the real engine, and what it costs in quality.
brew install usetrove/trove/troveFree and open source. Nothing leaves your machine. Run trove on your own sessions and see your own numbers.
The headline numbers
72.0%estimated
Median saving on 7 scripted workflows for an agent or script that sends no cache markers (range 51.6% to 79.8%). Trove switches prompt caching on for it.
19.3%estimated
Median on 3 scripted output-heavy sessions (range 2.5% to 50.7%). A monorepo test loop reaches 50.7%.
698 of 698measured
Failure and error lines still visible to the model after trimming.
Scripted sessions, replayed through the real engine with no API calls. They are not a forecast of anyone's bill. Claude Code already caches its own prompt, so the no-cache figure does not apply to it.
Does it cost quality?
Every check below is measured, not estimated.
698 of 698measured
Failure and error lines still visible to the model.
400 of 400measured
Requests kept the cached prefix byte-identical. 0 broken invariants; repeat runs byte-identical.
90 of 90measured
Aider tasks passed with Trove's optimizers on and with them off.
2.54 msmeasured
Engine time per request at p50 (p99 17.09 ms), measured in-process on one machine, excluding the gateway's forwarding time.
You don't have to take our word for it
Benchmarks say what Trove does on these sessions. Your sessions are what matter. So every fix starts in shadow mode, where it runs on a copy of your requests and records what it would have saved, minus its own overhead. It turns on only when that evidence is there, only between sessions, and it rolls itself back if the net saving goes negative.
Share of the bill saved
Trove shrinks repeated and noisy tool output without disturbing the part of the prompt your provider already has cached, and switches prompt caching on for agents that do not use it.
Median per kind of session. Everyday sessions already run on a cached prompt, so they leave the least to trim. The 94-turn feature session saves 8.6%, the most of any everyday workflow.
Scripted everyday sessions, replayed through the real engine
Short benchmark tasks cannot show a long working day, so trove bench workflows replays hand-scripted Claude Code sessions through the real optimizer engine, offline: no API call, no spend. How often a file is re-read, how noisy a shell is and how long a session runs are assumptions, so these figures say what the optimizers do on these patterns. They are not a forecast of anyone's bill. Model claude-sonnet-5-5, list prices as of 2026-11-08.
| Kind of session | Median saving (range) | Label |
|---|---|---|
| Everyday: fix a failing test, rename across a codebase, debug with logs, explore a repo, a ~90-turn feature session (7 workflows) | 3.7% (0.0% to 8.6%) | estimated |
| Output-heavy: a verbose test loop, a failing container build, incident logs (3 workflows) | 19.3% (2.5% to 50.7%) | estimated |
| A custom agent or script that sends no cache markers (the 7 everyday workflows again) | 72.0% (51.6% to 79.8%) | estimated |
- Everyday. Claude Code already caches its own prompt, so the optimizers can only trim what it sends. Even an oracle that deleted every tool output would save between 6.5% and 56.9% depending on the session.
- Output-heavy. Chosen because large, repetitive tool output dominates them. A ceiling for that kind of work, never a typical day, never averaged with the everyday row. One workflow does most of the lifting (a monorepo test loop at 50.7%); incident logs save only 2.5%.
- No caching. This saving is Anthropic's prompt caching, which Trove switches on for agents that do not use it. It is zero for an agent that already caches and does not apply to Claude Code.
Saving per workflow (estimated)
Everyday
Output-heavy
Custom agent, no cache markers
Full per-workflow table
| Workflow | Tier | Turns | Cost off (est.) | Cost on (est.) | Saving (est.) | Key lines kept | Prefix kept |
|---|---|---|---|---|---|---|---|
| quick-question | everyday | 3 | $0.0635 | $0.0635 | 0.0% | 0/0 | 3/3 |
| fix-failing-test | everyday | 12 | $0.1418 | $0.1418 | 0.0% | 10/10 | 12/12 |
| debug-with-logs | everyday | 12 | $0.2730 | $0.2628 | 3.7% | 70/70 | 12/12 |
| dependency-upgrade | everyday | 10 | $0.1594 | $0.1513 | 5.1% | 10/10 | 10/10 |
| rename-across-codebase | everyday | 28 | $0.5567 | $0.5566 | 0.0% | 8/8 | 28/28 |
| explore-and-review | everyday | 20 | $0.4750 | $0.4416 | 7.0% | 0/0 | 20/20 |
| long-feature-session | everyday | 94 | $3.5912 | $3.2806 | 8.6% | 80/80 | 94/94 |
| ci-failure-triage | heavy | 15 | $0.4420 | $0.2181 | 50.7% | 4/4 | 15/15 |
| container-build-debug | heavy | 14 | $0.2675 | $0.2159 | 19.3% | 66/66 | 14/14 |
| incident-log-forensics | heavy | 13 | $0.4405 | $0.4295 | 2.5% | 276/276 | 13/13 |
| quick-question (no caching) | custom agent | 3 | $0.1234 | $0.0597 | 51.6% | 0/0 | 3/3 |
| fix-failing-test (no caching) | custom agent | 12 | $0.6819 | $0.1986 | 70.9% | 10/10 | 12/12 |
| debug-with-logs (no caching) | custom agent | 12 | $0.7231 | $0.1517 | 79.0% | 57/57 | 12/12 |
| dependency-upgrade (no caching) | custom agent | 10 | $0.4826 | $0.1577 | 67.3% | 10/10 | 10/10 |
| rename-across-codebase (no caching) | custom agent | 28 | $2.2186 | $0.4814 | 78.3% | 8/8 | 28/28 |
| explore-and-review (no caching) | custom agent | 20 | $1.4643 | $0.4097 | 72.0% | 0/0 | 20/20 |
| long-feature-session (no caching) | custom agent | 94 | $16.4598 | $3.3321 | 79.8% | 89/89 | 94/94 |
How it was measured
Hand-scripted Claude Code sessions were replayed offline through the real optimizer engine, with no API calls. Two pre-registered studies ran the gateway on real tasks (Aider polyglot and SWE-bench Verified).
Where things stand. Scripted sessions show Trove helping where there is real waste: output-heavy work and agents that do not cache. On short real tasks, two pre-registered studies found no saving that can be told apart from noise. Trove makes no savings promise.
Real-task studies, decision rule and caveats
Real tasks, two pre-registered studies
Does the gateway with its safe optimizers on cost less than the same gateway with every optimizer off? Costs are estimated (list price on the token counts the agent reports). Pass or fail is measured.
| Measure | Aider polyglot, 30 tasks | SWE-bench Verified, 35 tasks |
|---|---|---|
| Study | gateway-cost-2 | swebench-gateway-2 |
| Model, agent | claude-haiku-5-5, Claude Code 2.1.318 | claude-sonnet-5-5, Claude Code 2.1.319 |
| Runs | 180 (90 pairs) | 140 (35 pairs per comparison) |
| Median paired cost change, all_safe vs passthrough (estimated) | −3.0% (95% interval −9.5% to +3.7%) | +4.4% (95% interval −7.0% to +13.8%) |
| Success (measured) | 90 of 90 under both | 28 of 35 vs 29 of 35 |
| Mean cost per run (estimated) | about $0.0026 | about $0.02 to $0.03 |
The two studies point in opposite directions, which shows how wide the noise is. In the SWE-bench study the two single optimizers were also within noise: noise_normalizer 0.0% (−11.0% to +20.0%) and duplicate_result_collapser +1.9% (−11.6% to +26.0%). The gateway's own records say the optimizers had almost nothing to remove: the median SWE-bench session was 5 turns, and all_safe removed about 0.2% of input tokens (estimated). A saving of that size cannot move a bill.
Decision rule
Trove recommends an optimizer for automatic use only if its cost reduction is at least 5% and the whole 95% interval lies at or below −5%, with the success rate not falling by more than 5 points and at least 30 pairs. Neither real-task study met that bar, so Trove does not claim a saving on real tasks yet. A result is never rounded up.
Caveats
- Two models and two Claude Code versions across the real-task studies; results may differ for other models, versions or longer sessions.
- Costs are list-price estimates, never an invoice.
- Token counts in the scripted runs use the engine's bytes-per-token ratio, not provider counts.
- The heavy tier was written to be waste-rich.
- The SWE-bench study ran on a separate machine and its result files are not yet in the repository.
Reproduce
python -m trovebench fetch polyglot -n 30 --seed 20261108
python -m trovebench run --study studies/gateway-cost-2.yaml --tasks <that folder> --batch <folder> --yesbash bench/scripts/95_workflows.shRead the methodology and the published workflow reports.