Intelligence note

An audit is a photograph of a market that keeps moving

Three separate things change your AI bill after the optimisation work is finished: the market, your own stack, and the setup you never touched. Only the first is widely accepted, and the third is the one nobody is watching.

Analysis, not established cause7 August 2026 · 7 min read

Analysis, not established cause. The pattern is real and measured; the explanation offered for it is a hypothesis and is not established.

The standard shape of an AI cost engagement is connect, analyse, recommend, leave. It is a defensible shape for a market that holds still. The question this note asks is narrow and answerable: does this one hold still, and if not, along how many independent axes does it move?

Three. They are genuinely independent, in the sense that any one of them can move your bill while the other two do nothing. They also differ sharply in how well we can prove them, and that difference is the honest part of this note.

One: the market moves, and this is the easy one to prove

Published prices change on a cadence measured in weeks, not years, and the changes are not drift — they are responses. A cheaper entrant forces a price cut; a flagship is quietly replaced by a stronger model at the same list price, which is a price cut expressed as capability rather than dollars. Our own price ledger is append-only precisely so this is a matter of record rather than recollection: every move we have ever observed is still there, with its date and its magnitude.

The consequence for a one-time audit is not that its recommendations were wrong. They were right on the day. It is that the ranking they were derived from is regenerated by the market every few weeks, and nothing about the audit updates when it does.

Two: your own stack grows into a larger bill

As a team's use of AI matures, the work it asks of a model tends to get harder: longer context, more retrieved material, more tool calls per task, more steps before an answer. Spend per task rises without any provider changing anything and without anyone making a decision that looks like a spending decision.

We have not measured this on our own customer base, and we are not going to imply that we have. Our production usage history is not yet long enough to separate a maturing workload from a growing one, and a chart that cannot tell those apart is not evidence. Treat this front as reasonable and widely observed, not as something we have proven.

Three: the setup you never touched changes underneath you

This is the front almost nobody watches, because everything visible about it stayed constant: same model identifier, same prompt, same code path. What can change is how many tokens the provider bills to answer it — through a silent update behind a stable model id, a shift in how much a reasoning model thinks by default, or a scaffolding change in the layer between your request and the weights.

The mechanism here is not speculative, and we have the artifact. A thinking model returned a one-word answer and billed sixty-eight output tokens. The billed number lives in a field that has nothing to do with the answer, so the answer can stay identical while the bill moves.

Exhibit AThe billed number and the visible answer are different fields

"usageMetadata": {
  "promptTokenCount": 7,
  "candidatesTokenCount": 1,
  "thoughtsTokenCount": 67,
  "totalTokenCount": 75
}

Captured through the CostMyAI connector on a live generativelanguage.googleapis.com call, 7 August 2026, and pinned as a regression test the same day. Reproduced from the earlier note that first documented it; the mechanism, not the incident, is the point here. Verifiable in the repository at src/lib/ingest/__tests__/dispatch-109.test.ts.

What this exhibit proves is that the bill for an unchanged request is free to move without the answer moving. What it does not prove is that it did move for a given workload over a given period. That is a different claim, it needs the same task measured months apart, and we did not have those measurements.

So we started taking them

From this month, eight fixed tasks run against six pinned models on the first of every month, and the token counts each provider reports are written to an append-only log. The tasks are frozen in source: a prompt may not be edited in place, and the fingerprint of the exact text sent is recorded on every row, so a prompt that was quietly changed shows up as a changed fingerprint rather than as drift. Failed calls are recorded as failures, so a gap in a series always states its reason.

Exhibit BThe instrument, as scheduled

job          task-drift            monthly, 02:30 on the 1st
tasks        8 frozen prompts      classification, extraction, summary,
                                   rewrite, SQL, arithmetic, planning,
                                   single tool-call decision
models       6 pinned ids          3 Google, 3 OpenAI
records      prompt + completion + reasoning tokens, upstream cost,
             latency, SHA-256 of the prompt and of the reply
storage      append-only; no row can be edited or deleted

The pinned set as committed on 7 August 2026. Two vendors, not the whole market: this is the set we can call on a fixed schedule with first-party credentials, and we would rather state the limit than imply coverage we do not have. Verifiable in the repository at src/lib/drift/tasks.ts.

There is no detector attached to it, and that is deliberate. One reading is not a series. A comparison written today would be a comparison nobody could test against real history, and shipping the alarm before the readings is how tools end up reporting movement they cannot substantiate. The meter runs; the comparison ships when there is something to compare.

What each front actually justifies

The market movesProven from our own append-only price ledger. Justifies continuous monitoring on its own, without any of the rest.
Your stack maturesReasoned, not measured by us. Plausible and widely reported, and we will not cite it as our finding until our own history is long enough to separate maturation from growth.
The setup changes silentlyMechanism proven, longitudinal case not yet ours. The exhibit shows the bill can move with the answer unchanged; the meter above is what will eventually show whether it does.

One front is enough to answer the question. If prices move monthly and provably do, a recommendation set derived from last quarter's prices is a historical document, however good it was when it was written. The other two fronts do not change that conclusion. They change how much of it you can currently see.

Every figure we publish states its source, its date, and what it does not cover.

How every figure is computed

The figures behind this note are
public, and frozen monthly.