A $20 prototype can turn into a four-figure monthly bill faster than most founders expect. LLM API pricing isn’t hard to understand, but headline rates never tell the whole story.
Your actual cost depends on prompt length, response length, repeated context, retries, and external actions. I don’t treat provider rates as a buying decision by themselves. I use a value score, meaning successful customer outcomes relative to fully loaded spend, to judge quality-adjusted economics.
Estimate the cost of a real customer request before comparing rate cards.
LLM API pricing starts with token math
Language model APIs charge for tokens processed on the way in and generated on the way out. Input tokens cover your system prompt, user message, chat history, retrieved documents, tool results, and other context. Output tokens cover the model’s response, while reasoning models may generate additional internal reasoning tokens.
Prompt and response tokens have different costs
Most providers charge more for generated text. That matters because a long response can cost more than a detailed prompt.
A support assistant that reads a 3,000-token account history and writes a 150-token reply has a different cost profile than a content tool that writes a 2,000-token draft. Don’t estimate both workflows with the same average request size.
Context also compounds. Every old chat message you resend becomes billable input again. A conversation that starts cheap can get expensive when you keep appending history without a retention rule.
The basic calculation is simple
For a standard non-cached request, use this formula:
Monthly model cost = (input tokens / 1,000,000 x input rate) + (generated tokens / 1,000,000 x output rate)
A cost calculator can apply these rates per million tokens to a real request.
If your provider supports caching, add cached input and cache-write rates as separate lines. If your app uses tools, retrieval, OCR, search, or browser automation, budget those services separately.
A low per-request cost means little if an agent retries that request five times after a tool failure.
Read model price cards without missing the expensive part
Price tables change often, so token pricing is only a starting point for any forecast. I recommend recording the exact model ID, rate-card date, context tier, and discounts in a pricing registry. “We use Claude” or “we use GPT” isn’t enough for finance or engineering.
The following rates are standard short-context text prices per million tokens, checked in August 2026. They’re useful reference points, not a substitute for checking live provider documentation before deployment.
| Model | Input | Cached input or cache hit | Output |
|---|---|---|---|
| OpenAI GPT-5.6 Luna | $0.20 | $0.02 | $1.20 |
| OpenAI GPT-5.6 Terra | $2.00 | $0.20 | $12.00 |
| OpenAI GPT-5.6 Sol | $5.00 | $0.50 | $30.00 |
| Claude Haiku 4.5 | $1.00 | $0.10 | $5.00 |
| Claude Sonnet 5 | $2.00 | $0.20 | $10.00 |
| Claude Opus 5 | $5.00 | $0.50 | $25.00 |
This table is a useful pricing comparison, but price alone doesn’t determine quality or total cost.
OpenAI, like many API providers, publishes separate standard and cached rates, batch discounts, and long-context pricing in its official API pricing documentation. Anthropic also separates standard input, cache writes, cache hits, and output in its Claude Platform pricing guide.
Long context can change the whole calculation
A large context window is a capacity limit, not a free place to store every document your customer has ever uploaded. Some models apply higher rates once prompts pass a context threshold.
Google’s Gemini Developer API pricing page also varies by model, modality, usage tier, and token type. If your product processes video, audio, or long files, don’t borrow a text-only estimate from another team’s use case.
Build a monthly API budget from real request patterns
A useful forecast starts with a representative week of production-like traffic. Measure request volume and token counts before optimizing, then define the value score as successful outcomes relative to fully loaded spend for each workflow. Guessing based on prompt length alone misses chat history, retrieved passages, and output variance.
Measure requests by workflow, not by product
Break usage into distinct jobs. A SaaS product may have onboarding chat, document extraction, support drafting, report generation, and code assistance. Each one needs its own token baseline and quality requirement.
For every workflow, track:
- Average and 95th-percentile input tokens per request.
- Average and 95th-percentile output tokens per request.
- Requests per active customer each month.
- Retry rate, fallback rate, tool-call count, and benchmark score.
- The model ID used for the final successful answer.
- The value score for each workflow, recorded beside usage and spend.
Because long documents and difficult conversations create bill surprises, compare 95th-percentile usage with the value score and retry behavior.
Work through one planning scenario
Assume 10,000 monthly requests. Each request sends 8,000 tokens and generates 1,200 tokens. That gives 80 million tokens sent and 12 million returned; use a cost calculator to validate separate input, output, and cached input lines.
At GPT-5.6 Luna’s listed standard rates, the inference cost is about $30.40 monthly: $16 for input and $14.40 for output. At GPT-5.6 Terra’s rates, the same traffic is about $304.
That tenfold monthly cost difference comes before quality, retries, and prompt reuse. The cheaper bill helps only when quality remains acceptable, as shown by its value score; frequent failures can erase its price advantage.
Costs that hide behind a normal-looking prompt
Most cost overruns don’t come from a single oversized answer. They come from a system that adds context or retries work without reporting it clearly.
Chat history and RAG inflate request size
A RAG request might include a 700-token system prompt, a 300-token question, four retrieved chunks of 600 tokens each, and prior conversation history. A larger context window gives you more room, but added context isn’t free. You can cross 4,000 input tokens before the model writes one word.
Retrieval quality matters here, since text embeddings and vector services can add to the RAG bill. Sending eight weak chunks “just in case” makes responses slower and more expensive. Better chunking, metadata filters, and reranking reduce irrelevant context before the model sees it.
For technical support products, keep warnings and exceptions beside the source passage that supports them. Splitting a policy rule from its limitation can create inaccurate answers and extra follow-up calls.
Tool calls, retries, and bad routing add up
Tool-using agents often make several model calls per customer action. A model may plan a search, call an API, read the returned data, revise its answer, then retry after a formatting failure.
Each step can add input, output, external API charges, and latency. Log the parent request ID across every sub-call. Otherwise, a $1 customer action can look like ten unrelated ten-cent requests. Include retries and failed sub-calls in its completed-task value score.
For a broader operational framework, see these rate limits and spend controls for AI agents. A token budget without sub-call limits only covers part of the risk.
Choose models by task quality, not unit price alone
The cheapest model API may not be cheapest if it misses your quality floor or creates costly failures. A pricing comparison should weigh token pricing against an acceptable price performance ratio, with the quality floor driving a value score. Inference cost covers only part of completed-task economics.
Set the quality floor before comparing rates
Build a small evaluation set from real, permission-safe requests. Test prompting, routing, and workflow design before taking on fine tuning work.
Include easy cases, long-context cases that test the model’s context window, ambiguous inputs, and failures that would matter to a customer.
Use it to produce a value score from correctness, formatting reliability, latency, safety behavior, and human-edit time. A benchmark score can help narrow a shortlist, but it cannot tell you whether a model understands your product’s terms, customer records, or source documents.
For code generation, the same rule applies. A low-cost model that writes brittle code may create costly review work. Include that effort in the value score, since it can dwarf the inference bill. This coding-focused LLM comparison is useful when code quality and developer workflow are part of the decision.
Separate low-risk and high-stakes workflows
Small, cheaper models often make sense for classification, extraction, routing, short rewrites, and structured transformations.
Reserve premium models, including reasoning models, for workflows that meet your value score threshold. These may involve difficult reasoning, sensitive customer communication, complex code changes, or tasks that trigger business actions.
I would not use a premium model for every request just because it performed well in a demo. I also would not put a weak model in front of customers because its token price looks attractive.
Cut spend with caching and model routing
Cost control works best when it is part of the architecture. Treat caching and routing as cost optimization, rather than trimming prompt text after the bill arrives.
Prompt caching for stable prompt prefixes
This approach is useful when many requests begin with the same long system instructions, product documentation, or policy text. Reusing that cached input is less useful when every prompt is heavily personalized.
Anthropic states that a cache hit costs 10% of the standard input price. Its five-minute cache write costs 1.25 times the standard input rate, so cached input pays off after one reuse within that window. OpenAI’s published cached-input rates follow a similar pattern of sharply discounted repeated input.
Keep dynamic user data outside the cached prefix. Cache keys that mix stable policy text with changing account information will miss often and can create privacy problems.
Route only when you can test the route
A practical model cascading pattern starts with a lower-cost model for defined tasks, then escalates when a confidence check, validation rule, or request type calls for more capability.
For example, document classification may stay on a small model. A request that needs detailed analysis, long source grounding, or a high-value account response can move to a stronger model.
Don’t route based on vague prompts like “use the best model when needed.” Write the routing rules and test their failure cases. Track a value score that measures whether escalation improves completed-task quality, not merely whether it reduces spend.
Managed APIs versus open-weight hosting
Using open weight models can reduce recurring vendor charges from API providers, but downloading the model isn’t the operating cost. Open source models may offer broader availability or customization, but open licensing doesn’t remove operating costs. Your team still owns compute, deployment, scaling, monitoring, security work, upgrades, incident response, and the engineering time behind them.
A free model file is not free inference
GPU capacity is often the largest expense, and inference cost rises when utilization is low. Serverless endpoints can handle uneven traffic without dedicated GPUs running constantly, but peak traffic still requires capacity planning. You also need ways to manage model versions, prompt changes, request queues, observability, and data retention.
If managed services have outages or rate limits, self-hosting gives you more control. It also moves responsibility to your team. For a two-person SaaS, that trade can be expensive even if the hourly GPU rate looks reasonable.
Hosting makes sense in narrower cases
Self-hosting is more plausible when usage is steady and utilization is high. It’s also a stronger fit when privacy, data residency, or retention requirements are strict and the team already knows how to run reliable inference infrastructure. Serverless endpoints may improve the overall choice when managed endpoint economics beat dedicated hosting for uneven traffic. Judge the value score using engineering time, reliability, maintenance, and flexibility, not the model file or hourly GPU rate alone.
Managed APIs are usually easier when traffic is unpredictable or the product is still finding its market. The same total-cost question applies to automation tools. A self-hosted workflow platform may remove a license fee, but maintenance still has a price, as this n8n review and pricing breakdown makes clear.
Put spend controls close to customer activity
A monthly provider budget is too blunt. One abusive account, broken integration, or runaway agent can drive request volume beyond safe limits before anyone notices.
Use budgets at several levels
Set limits per customer, workspace, workflow, and request. A document-analysis feature may deserve a higher ceiling than a free-form chat widget. Hard caps protect cash flow, while soft alerts give your team time to investigate.
Return a clear fallback message when a customer reaches a limit. Silent failures turn a cost-control problem into a support problem.
Track unit economics beside model telemetry
For every workflow, calculate cost per completed action, value score, cost per active customer, and cost as a share of revenue. Measure the first with monthly cost divided by active customers, and the second with monthly cost divided by revenue.
Keep api usage costs for model charges separate from text embeddings, vector storage, tool APIs, and infrastructure.
Review these figures after a provider price change, a model migration, or a prompt redesign. I would also revisit the rate card on a 60 to 90-day schedule. API providers change model names and discounts too quickly. A one-time spreadsheet can’t stay reliable, so maintain a pricing registry with current rates and discount terms.
A practical buying process for small teams
Use this sequence before committing to a provider or model family:
- Define the user job and identify the failure that would cost you the most. Write down the success condition and who reviews edge cases.
- Collect representative requests and remove sensitive data before evaluation. Include common, difficult, and failure-prone examples.
- Test two or three models against the same task set. Record quality, latency, token use, retry rates, benchmark score, and pricing comparison.
- Forecast a low, expected, and high-usage month using request volume as a key scenario input. Use a cost calculator to validate each scenario, with separate input, output, cache, batch discounts, and tool-cost lines. Record the resulting monthly cost.
- Set caps and alerts before launch, not after the first surprise invoice. Tie them to customers, teams, and usage patterns where possible.
- Keep a fallback plan for model downtime, rate limits, or a sudden pricing change. Document when to switch and who owns that decision.
The right choice might be a low-cost model for most requests and a more capable fallback for a small percentage. Judge that combination by its value score, based on completed-task economics rather than unit price alone.
Conclusion
The useful LLM API pricing number isn’t the rate shown on a pricing page. It’s the monthly cost of a successful customer outcome after context, retries, tools, and quality checks.
Start with real request traces, choose models by workflow, and assign each path a value score. Set limits around every expensive path, then review whether spending supports successful customer outcomes. Predictable AI costs come from disciplined architecture, not from picking the lowest rate-card number.
FAQ
How do I calculate LLM API cost from tokens?
To estimate monthly cost, multiply monthly input tokens by the provider’s input rate per million. Do the same with output tokens and the output rate. Add both totals, plus cached input charges, cache-write fees, embedding costs, tool APIs, and hosted infrastructure.
What is the cheapest LLM API for high-volume text generation?
There is no universal answer. A low-cost model fits when it reliably completes your specific task without retries or human cleanup. Test it against your own inputs before treating its list price as a saving.
Does prompt caching reduce API costs?
Yes, when requests reuse a stable prompt prefix. It helps with long system instructions, shared documentation, and repeated policy content. It won’t help much when each request contains different user data.
Are open-weight LLMs free to run?
No. You may not pay a token fee to a model provider, but you still pay for GPU capacity, storage, networking, deployment, scaling, monitoring, security, and engineering time.
