What AI
really costs.
AI is sold by the token, and the price per token is right there on every provider's pricing page. But that number is almost never what a job actually costs you. Understanding the gap between the sticker price and the real bill is what keeps an AI budget from quietly spiraling.
Everything a model does is measured in tokens — little chunks of text, roughly ¾ of a word each. Every provider charges a price per million tokens, and because inference is paid on every single request, tokens are the meter your whole AI bill runs on. So let's follow the money.
You pay on both sides — unevenly
Every request has two token counts. Input tokens are everything you send in — your prompt plus any documents, history, or context. Output tokens are what the model writes back. You're billed for both, but they're not priced the same: output almost always costs several times more than input, because generating text is far more work than reading it.
A pricing page like "$3 per million input / $15 per million output" is telling you output is 5× pricier here. That's why a chatty model that writes long answers can cost far more than its input price suggests — the expensive side is the side it can't stop talking on.
Cheaper per token ≠ cheaper per task
Here's the mistake almost everyone makes: comparing models by price per token. The problem is that different models use different numbers of tokens to finish the same job. A model that's half the price per token but writes twice as much — or gets it wrong and needs a second try — can quietly cost more to complete the actual task.
Watch a realistic head-to-head. The "cheaper" model has the lower sticker price and still ends up with the bigger bill:
Not every token is worth the same
Tokens aren't a commodity — a barrel of oil is interchangeable with any other barrel, but one model's token can carry far more useful thinking than another's. This is token efficiency (sometimes "intelligence density"): how much real work a model does per token. A more efficient model reaches the right answer with fewer tokens, so it can be cheaper even at a higher price per token.
Two things quietly inflate token counts. Verbosity — some models just ramble. And reasoning — models that "think" step by step before answering spend extra tokens doing it. That thinking is worth paying for on a hard problem and pure waste on an easy one, which is why matching the model to the job is the whole skill.
Cheaper AI means more AI, not less
There's a counterintuitive force worth knowing, because it shapes the whole industry. When something gets cheaper to use, people don't spend less — they use far more of it. This is Jevons paradox. As the cost per token falls, AI gets baked into more products, more workflows, more requests — so total demand for chips, electricity, and data centers goes up, not down.
Falling token prices don't shrink your AI footprint automatically — they invite you to do more with AI. The savings are real, but they get spent on doing more, unless you're deliberate about it.
How to actually spend less
Once you think in cost-per-task, the levers are obvious:
- Right-size the model (routing). Send easy work to a small fast model; save the expensive one for the hard, high-judgment steps. This is the single biggest lever.
- Shrink the input. Don't dump whole documents into every request — use retrieval to pull in only the relevant snippets, and caching to avoid re-paying for repeated context.
- Cap the output. Ask for the format you actually need (three bullets, not three paragraphs). Output is the pricey side.
- Or leave the meter behind. If your volume is high enough, running an open model on your own hardware replaces the per-token bill with a flat cost you control.
Designing that routing, retrieval, and local-vs-cloud mix so it's cheap and good is most of what an AI build actually is. Let's talk →
Common questions
What's the difference between input and output tokens?
Input tokens are everything you send the model (your prompt plus any context or documents); output tokens are what it generates back. You pay for both, and output usually costs several times more than input because generating text is more expensive than reading it.
Why is cost-per-token misleading?
Two models can advertise very different prices per token but use different numbers of tokens to finish the same job. A model that's cheaper per token but rambles — or gets it wrong and needs a retry — can end up more expensive per task. The metric that matters is cost per completed task.
What is token efficiency?
Token efficiency (or "intelligence density") is how much useful work a model does per token. An efficient model reaches a correct answer with fewer tokens. Reasoning models that think step by step spend extra tokens — worth it for hard problems, wasteful for simple ones.
What is Jevons paradox in AI?
It's the idea that when something gets cheaper to use, people use much more of it. As AI tokens get cheaper, usage grows rather than shrinks — increasing total demand for chips, energy, and data centers even as the price per token falls.