Stop paying thousands for the same questions.

Illustration: AI-generated, art-directed by the author.

There is a date circled on my calendar: September 1, 2026. That is the day Anthropic's standard pricing for its mid-tier workhorse model rises 50%, from $2 to $3 per million input tokens, as its introductory pricing ends. Nobody using these tools gets to ignore cost engineering after that.

The industry has noticed. In the FinOps Foundation's 2026 survey of the people who manage cloud spending, covering more than $83 billion of it, 98% now manage AI costs. Two years earlier it was 31%. That is the fastest a cost category has ever gone from curiosity to everyone's problem.

Here is what most of those teams have not yet internalized: an AI bill is not like a cloud bill. It is mostly the same tokens, bought again and again. And the providers will sell you those repeats at up to 90% off, if you ask in the right shape.

Lever one: stop paying full price for repetition

Every serious AI workload has a fixed part and a fresh part. The fixed part is the system prompt, the product catalog, the policy document, the knowledge-base passages that ride along with every request. The fresh part is the user's actual question. In the systems I run and audit, the fixed part is routinely 10 to 100 times larger than the fresh part.

Prompt caching bills those parts differently, and the published discounts are not subtle:

• OpenAI: cached input on GPT-5 costs $0.125 per million tokens versus $1.25 fresh. 90% off.
• Anthropic: cache reads cost 10% of the base input price. The same passage that costs $5 per million tokens fresh costs $0.50 as a cache hit.
• Google Gemini: cached tokens are also 90% cheaper, with a catch the others do not have: cache storage bills by the hour ($1.00 to $4.50 per million tokens per hour). Caching a corpus you rarely hit can cost more than not caching it.

The mechanics matter less than the mindset: if your assistant reads the same 200 pages on every request, you are choosing, every day, whether to pay for those pages 500 times or once. Structuring prompts so the stable part stays stable is boring engineering worth actual money.

You are choosing, every day, whether to pay for the same 200 pages 500 times or once.

Lever two: stop sending a surgeon to take temperatures

The second spread hides inside each provider's own menu. Anthropic's small model takes input at $1 per million tokens; its top-priced takes $10. OpenAI's mini model is $0.25 against $1.25 for the flagship. That is a 5 to 10x price difference sitting in a dropdown.

Most requests in a real workload are small: classify this, extract that, summarize this paragraph. Routing them to the small model and saving the big one for judgment calls is the single largest lever most teams have not pulled. In my own stack, every generation is traced with its cost, and the pattern is always the same: the flagship model was doing commodity work most of the day.

Lever three: know your cost per task, because the meter changes shape

Two vendor-documented gotchas make "we'll just watch the monthly bill" a losing strategy:

First, the price-per-token can change under you, as the September 1 increase shows. Second, and sneakier: the tokens themselves can change. Anthropic documents that its newer models use a tokenizer that produces roughly 30% more tokens for identical text. Migrate models and your consumption rises a third with zero change in usage. Teams that only watch the monthly total will misdiagnose that as growth.

The unit that survives both distortions is cost per task: what does one answered ticket, one drafted email, one processed document actually cost? Trace it per request (tools like Langfuse make this a config change, not a project) and the number becomes manageable like any other unit economic. My whole operation runs on this: I do not know what "AI" costs me per month until I look; I always know what a published post or an answered call costs.

Cost per task and cache hit rate make you the adult in the room. A monthly total makes you the topic.

The real message under the arithmetic

Notice what all three levers have in common: none of them is a discount code. They are all forms of the same discipline, which is refusing to re-send what the model already has, re-think what a smaller brain could handle, or re-guess what a traced number could tell you. (The same discipline applied to your knowledge base, deciding what deserves to be remembered at all, is curating your corpus. So is the question of whether to stop renting the brain entirely and own it.)

This is the unglamorous half of building a tool that is actually fit for one job rather than a licence handed to everybody. Nobody in procurement decides your cache strategy or your routing table. A specialist does, per workload, and that is most of the difference between an assistant that pays for itself and one that quietly does not.

AI spend is about to be a line item every CFO interrogates, and the September price hike will start those meetings. Walk in holding a cost per task and a cache hit rate, and you are the adult in the room. Walk in holding a monthly total, and you are the person the meeting is about.

Your AI bill is not a cost of doing business. It is a scorecard of how often you pay for the same thing twice. What is your cache hit rate? If the answer is "what's a cache hit rate," the scorecard says zero.