Krypton
Case study

What we measured, and what it taught us

The guides tell you what Krypton does. This page keeps the experiments behind them: one open-source project, garth, tested in depth, and 19 more repositories for the prompt-caching survey.

35%
cheaper promptsgarth, same prompts, live Claude API, stable files first.
12 of 20
repositories helpedOrdering lost on 7, which is why it checks first.
16.9%
cheaper later changesAfter a cleanup, averaged over six runs; the typical run saved about 6%.
Read these as examplesEvery number here comes from the repositories named on this page. Your own result depends on your repository's history and size.

Prompt caching on the live Claude API

We replayed 12 real commits of garth (about 40,000 tokens of source per turn as Claude counts them) against the live Claude API (Sonnet 5.5, four cache breakpoints per request) and measured how much of each prompt the provider reused and what the replay cost.

File orderPredicted reuseMeasured reuseCost of the 13 turns
Krypton (most stable first)73%71%$0.51
Alphabetical48%47%$0.79
Reverse (most volatile first)0%0%$1.30

Excluding the first turn, which can never hit the cache. Including it, Krypton's order reused 65.7%. Krypton's order made the replay about 35% cheaper than alphabetical order. With a single cache breakpoint at the end of the prompt, every order reused 0%, so the ordering only pays when the stable files sit in front of earlier breakpoints. Predictions come from the shared prefix of consecutive prompts and landed within about two points of the live numbers.

Does it hold on other repositories?

Not always. We ran the same replay on a second repository, pluggy, against the live Claude API, and Krypton's order did worst:

File orderPredicted reuseMeasured reuse
Krypton (most stable first)4%4%
Alphabetical23%22%
Reverse (most volatile first)26%25%

The live measurements on garth and pluggy were within one point of the predictions. The table compares predicted reuse across 20 repositories, ranked by Krypton's gain over alphabetical order. These results show why Krypton evaluates several candidate orders against repository history: it chooses the strongest candidate only when the replay predicts an improvement over your current order. If no candidate clears that bar, the request keeps its original order.

RepositoryFilesKryptonAlphabeticalReverseKrypton vs alphabetical
pallets/flask2451%20%18%+31
pallets/click1727%0%32%+27
matin/garth4173%48%0%+25
pallets/itsdangerous845%29%17%+16
pallets/jinja2519%6%17%+13
psf/requests1923%12%31%+11
encode/httpx2327%16%33%+11
Textualize/rich10049%38%21%+11
pallets-eco/flask-wtf1042%32%14%+10
hynek/structlog2119%14%28%+5
pypa/packaging2235%33%37%+2
tqdm/tqdm3122%20%40%+2
pallets/markupsafe233%33%33%+0
agronholm/anyio4640%41%32%−1
theskumar/python-dotenv833%36%27%−3
python-attrs/attrs1911%14%43%−3
python-jsonschema/jsonschema3714%17%37%−3
python-attrs/cattrs5025%37%38%−12
pallets/werkzeug5423%36%11%−13
pytest-dev/pluggy84%23%18%−19
How the guard works. Krypton compares several candidate orders using the repository's history and picks the one with the highest predicted cache reuse. It changes your order only when that candidate is expected to do better; otherwise, the request goes out in the order you already chose. You can check your own repository with krypton order . --subdir src --model gpt-4o --post. It replays recent commits against the candidate orders without running a model or sending source.

Does the ordering guard pick well?

We tested the decision on garth without peeking: the decision is made from history alone, then the next 12 commits are replayed with exact token prefixes. This was repeated from 14 starting points, and for three kinds of caller order.

Your orderReorderedGain when reorderedReordered and worseKept but would have gained
Alphabetical13 of 14+13.4 points11
Shuffled10 of 14+27.1 points04
Worst case (most volatile first)14 of 14+47.9 points00

Two things to read from it. When your order is already poor, it reorders and the gain is large. When it hesitates, it keeps, which in the shuffled case means missing some gains: the guard is deliberately conservative, because a missed gain costs nothing and a bad reorder does.

Refactoring and the payback

A Krypton-guided refactor of garth took the logic-complexity flags from 9 to 0 with all 163 tests passing, where an unguided refactor left 8 of 9. We then asked whether the cleaned code is cheaper for an agent to change. Fresh agents made the same five change tickets on the original code and after each of two cleanups, six runs each, with real token usage priced at list prices:

Changes after the cleanupNet dollars savedReturn on total spend
10−$0.40−94.0%
25−$0.29−27.5%
66 (break-even)+$0.00+0.1%
100+$0.25+5.8%
250+$1.34+12.5%
500+$3.15+14.7%

Return on total spend includes the one-time cleanup, so it approaches 16.9% as changes accumulate. The table uses the clean-everything averages.

A CI gate over twenty revisions

We also tested Krypton as a check in CI. One feature of garth grew over 20 sequential revisions made by an agent, three times with each setup. In the plain setup the agent just did each ticket. In the gate setup the harness scored the code with Krypton after each ticket, and if a function the ticket touched was over the structure limit, an agent was sent back with Krypton's report to fix it before the next ticket. Neither agent was given the Krypton tool or told to refactor.

PlainWith the gate
Cost of 20 revisions$0.92$1.49 (+62%)
Worst function at the endabout 1,900× over the limitabout 12× over
Structure score at the end5572
Revisions done correctly60 of 6060 of 60