Cost per Token Explained, LLM Pricing and What to Know

Cost per token sits at the center of LLM pricing, but it is also one of the most misleading numbers on an AI pricing page. A model with a low token rate can still become expensive once you factor in long prompts, output-heavy responses, retrieval context, retries, and multi-step agent workflows. That is why the real question is not “What does this model charge per million tokens?” but “What will this workflow cost to run in production?”

In simple terms, cost per token is the rate an AI provider charges for processing input and generating output. In practice, though, token based pricing only explains part of the bill. Actual spend depends on how many tokens each request uses, how often context gets repeated, whether cached prompts are available, and how many model calls sit behind a single user action.

What Is Cost Per Token?

Cost per token is the amount an AI provider charges for the text an LLM processes and generates.

A token is a small unit of text. Depending on the tokenizer, a token may be a full word, part of a word, punctuation, or a short character sequence. In English, one token is often close to 0.75 words, but the exact count varies by language and model.

Most AI models do not charge per request. They charge per token, usually shown as a price per 1 million tokens. In practice, your total cost usually comes from two categories:

• input tokens, which are the tokens you send to the model
• output tokens, which are the tokens the model generates in response

If a model costs $2 per 1 million input tokens and $8 per 1 million output tokens, the total bill depends on how much text goes in and how much comes out.

Why Token Based Pricing Exists

A flat price per request would not work well for LLM usage because requests vary widely. One prompt might ask for a quick classification with 100 tokens of input and a 20-token answer. Another might include a long document, a large system prompt, retrieved knowledge-base chunks, and a 1,500-token output. These requests place very different loads on the model.

Token based pricing gives providers a usage-based model that scales across different workloads such as:

• chatbots
• document summarization
• coding assistants
• research agents
• retrieval-augmented generation workflows
• AI-powered search and support tools

How LLM Pricing Works

Most LLM pricing follows the same basic structure, even though providers present it differently.

Input tokens

Input tokens include everything you send to the model, such as:

• the user prompt
• system instructions
• conversation history
• retrieved context from a RAG pipeline
• tool outputs or structured data passed back into the prompt

If your request includes a 3,000-token system prompt, 2,000 tokens of retrieved content, and a 500-token user message, all of that counts toward input usage.

Output tokens

Output tokens are the tokens the model generates in its response. A short answer costs less than a long report, code block, or multi-step explanation. This matters because output pricing is often higher than input pricing. A workflow with short prompts but long answers may cost more than expected.

Cached input tokens

Some providers support prompt caching. This reduces the cost of repeated prompt prefixes, such as long system prompts or repeated context that stays the same across many requests. Cached token pricing matters in products with:

• long system prompts
• recurring instructions
• repeated document context
• multi-turn workflows with stable setup text

Additional usage charges

Modern AI pricing is no longer only input versus output. Depending on the provider, you may also see charges tied to:

• reasoning-heavy models
• multimodal inputs such as image or audio
• tool use, web search, or agent actions
• vector storage or retrieval layers
• batch jobs and asynchronous processing

That is why the price shown on a model card is only the starting point.

Cost Per Token Formula

The basic formula for a text request is simple:

Request cost = (input tokens ÷ 1,000,000 × input price) + (output tokens ÷ 1,000,000 × output price) Here is a quick example.

Assume a model is priced at:

• $2 per 1 million input tokens
• $8 per 1 million output tokens

Now imagine a request with:

• 5,000 input tokens
• 1,000 output tokens

The calculation is:

• input cost = 5,000 ÷ 1,000,000 × $2 = $0.01
• output cost = 1,000 ÷ 1,000,000 × $8 = $0.008

Total request cost = $0.018

That looks tiny on its own. The problem starts when the same workflow runs hundreds of thousands of times each month.

Real LLM Pricing Examples

Token pricing changes frequently as providers update models and release new versions. Looking at actual marketplace pricing gives a clearer picture of how widely costs can vary between models. The examples below use publicly listed rates available through Tokenware at the time of writing.

Model	Input Price (1M Tokens)	Output Price (1M Tokens)	Typical Use Case
GPT-5 Nano	$0.05	$0.40	Classification, tagging, routing
GPT-4o Mini	$0.15	$0.60	Customer support, chat assistants
DeepSeek V4 Flash	$0.09	$0.17	High-volume automation
GLM 5	$0.51	$1.54	General-purpose AI applications
Kimi K2.6	$0.56	$2.31	Long-context reasoning
Claude Haiku 4.5	$0.29	$1.43	Fast responses at scale
Claude Sonnet 4.5	$0.86	$4.29	Writing, coding, research
GPT-4.1	$2.00	$8.00	Production assistants
Claude Opus 4.5	$5.00	$25.00	Advanced reasoning workloads
GPT-5 Pro	$15.00	$120.00	Premium reasoning tasks

The gap is large. On Tokenware, output pricing ranges from about $0.17 per million tokens for DeepSeek V4 Flash to $120 for GPT-5 Pro. It also shows a consistent pattern across providers: output tokens often cost far more than input tokens, which is why model choice and response length have such a direct impact on total AI spend.

Input vs Output vs Cached Tokens

Not all tokens cost the same, and that difference matters when you estimate spend.

Token Type	What It Includes	Why It Matters
Input tokens	Prompt, system instructions, chat history, retrieved context	Often the biggest cost driver in RAG and long-context workflows
Output tokens	The model response	Expensive in report generation, coding, and long-form answers
Cached input tokens	Repeated prompt prefixes or reused setup text	Lowers cost when the same instructions or context appear often

If your product sends long documents into the model, input cost may dominate. If your product generates long answers, summaries, or code, output cost may dominate instead.

Why the Cheapest Model on Paper May Cost More in Practice

This is where many teams misread AI pricing. A model with a lower listed price is not always the cheaper model in production. Real cost depends on what the workflow does after the request starts.

Output-heavy workflows distort cost

A model may have cheap input pricing but expensive output pricing. If your application generates long answers, the output side can dominate the bill.

Reasoning-heavy models do not behave like standard chat models

Some reasoning models spend more tokens thinking through a problem or produce much longer outputs. In practice, this can make a supposedly cheaper model cost more than a higher-priced competitor on the same task.

Agents multiply the number of requests

An AI agent may break one user task into multiple model calls for planning, retrieval, summarization, rewriting, and final response generation. The user sees one result, but the backend may have made five or ten billable requests.

Long context changes the economics

A model with a large context window is useful, but it becomes expensive if you keep replaying long chat histories or large document chunks on every turn. The lesson is simple: compare models by effective cost per completed task, not by the headline price per million tokens.

Cost Per Request Examples

Example 1: Support chatbot

A support assistant receives:

• 400 input tokens from the user
• 2,000 tokens of system instructions and policy
• 2,500 tokens of retrieved help-center context
• 600 output tokens in the final answer

Total tokens:

• input = 4,900
• output = 600

Using a model priced at $0.40 per 1M input tokens and $1.60 per 1M output tokens:

• input cost = 4,900 ÷ 1,000,000 × $0.40 = $0.00196
• output cost = 600 ÷ 1,000,000 × $1.60 = $0.00096

Total = $0.00292 per reply

That is less than one cent, but if you process 1 million support turns in a month, the cost becomes roughly $2,920 before any extra tooling or retries.

Example 2: RAG document assistant

A user uploads a long report and asks for a summary. The workflow sends:

• 12,000 tokens of source text
• 1,000 tokens of instructions and metadata
• 1,200 output tokens

Total tokens:

• input = 13,000
• output = 1,200

Using a model priced at $2 input and $8 output per 1M tokens:

• input cost = 13,000 ÷ 1,000,000 × $2 = $0.026
• output cost = 1,200 ÷ 1,000,000 × $8 = $0.0096

Total = $0.0356 per request

The model itself is not the only cost driver here. The long document is.

Example 3: Research agent

An agent performs five model calls to complete one task:

1. plan the task
2. search and rank sources
3. summarize results
4. refine the answer
5. write the final response

If each call averages 4,000 input tokens and 700 output tokens, the total for one task becomes:

• 20,000 input tokens
• 3,500 output tokens

At $2 input and $8 output per 1M tokens:

• input cost = $0.04
• output cost = $0.028

Total = $0.068 per completed task

That is the number that matters, not the price of any single prompt.

What Increases Your AI Bill in Production

The biggest mistake in cost planning is treating one request as one clean prompt. Real systems have more moving parts.

Long system prompts

Every rule, example, and instruction adds tokens. If you paste large prompt templates into every call, the bill rises quickly.

Chat history replay

In multi-turn chat, old messages are often sent back to the model each turn. Without summarization or trimming, the same content gets billed repeatedly.

Retrieval-augmented generation

RAG improves answer quality by inserting document chunks into the prompt. It also increases input token usage, especially when retrieval is noisy and too many chunks are passed through.

Long outputs

Reports, code, verbose assistants, and detailed explanations all increase output spend.

Retries and fallback models

If a request fails validation, triggers a fallback, or retries after a tool error, token usage climbs even though the user sees only one final answer.

Wrong model selection

Using a premium reasoning model for classification, extraction, or tagging often wastes budget. Many workflows do not need a flagship model.

Prompt Caching and Cached Token Pricing

Prompt caching is one of the easiest ways to lower cost in high-volume workflows, yet many teams ignore it. Prompt caching works when the provider recognizes repeated prompt prefixes and bills them at a lower rate than normal input tokens. This is useful for:

• large system prompts
• fixed instructions shared across users
• recurring templates in customer support, research, or document workflows

A simple example:

• normal input price: $2 per 1M tokens
• cached input price: $0.50 per 1M tokens
• repeated system prompt: 10,000 tokens used across thousands of requests

If that repeated prompt is cacheable, the savings add up quickly over time. Caching does not solve every cost problem, but it is one of the clearest levers in token based pricing, especially for apps with stable prompt structure.

API Pricing vs Chat Subscription Pricing

This is a common point of confusion. A chat subscription, such as a consumer or team plan, is not the same thing as API pricing. Subscription plans usually give access to a chat interface with plan limits, feature caps, and usage rules set by the provider. API pricing is separate and billed based on usage.

If you are building a product, chatbot, assistant, or internal tool, the number that matters is the API cost of the underlying model workflow, not the monthly price of a chat subscription. Do not estimate production LLM costs from a $20 or $200 subscription plan. Those are different products.

How to Estimate Monthly Spend

The best way to forecast cost is to measure the average cost of a completed workflow, not a single prompt in isolation.

Use this formula:

Monthly cost = average cost per completed task × number of tasks per month

For a more accurate estimate, break the workflow down into:

• average input tokens per request
• average output tokens per request
• average number of model calls per task
• percentage of traffic using premium models
• retry rate
• cache hit rate
• multimodal or tool usage charges, if applicable

Example:

• average support conversation cost = $0.04
• monthly support conversations = 80,000

Estimated monthly cost = 80,000 × $0.04 = $3,200

Then test what happens if you:

• cut prompt size by 20 percent
• move classification to a cheaper model
• reduce output length
• improve cache hit rate

That is how cost planning becomes useful instead of theoretical.

How to Reduce Cost Per Token Without Hurting Quality

Lower spend does not always require switching providers. In many cases, the biggest savings come from workflow changes.

Trim prompts

Keep only the instructions that improve results. Remove repeated guidance, old examples, and unnecessary context.

Summarize chat history

Do not resend the full conversation if a short summary preserves the important context.

Send fewer retrieved chunks

Improve retrieval quality so the model sees fewer, more relevant passages.

Match the model to the task

Use cheaper models for routing, classification, extraction, moderation, and simple summaries. Save premium models for difficult reasoning or high-stakes output.

Limit output length

If you do not need a long answer, set clear output constraints and tighter prompts.

Use caching where possible

Repeated prompt prefixes are one of the clearest opportunities to reduce input cost.

Track spend by workflow

Measure cost by feature, endpoint, or user action, not only at the account level. This is the fastest way to find expensive prompts and wasteful agent flows.

Common Mistakes When Comparing LLM Pricing

Most pricing mistakes happen when teams compare model rates without looking at the full workflow behind each user action. A model may look cheap on paper, then cost more in production because the app keeps resending chat history, adding retrieval context, generating long outputs, or making multiple backend calls for one task. Another common mistake is focusing on input token price while ignoring output pricing, even though output tokens are often much more expensive. The better way to compare LLM pricing is to measure cost per completed task, not token price in isolation.

What to Check Before Choosing a Model

The lowest token price does not always produce the lowest AI bill. A model may look cheaper in a pricing table, then cost more in production because it needs longer prompts, generates longer answers, or triggers more retries and fallback calls. The right comparison is not price per million tokens alone. It is cost per completed task.

Start with the shape of the workload. If your product sends long documents, chat history, or retrieval context into the model, input pricing matters most. If it generates long reports, code, or detailed answers, output pricing matters more. Then look at how many model calls sit behind one user action. A single chatbot reply is one thing. An AI agent that plans, searches, summarizes, and rewrites before answering is another. The second workflow may trigger several billable calls even though the user sees one response.

You also need to look at prompt efficiency. A model that only performs well with long instructions, large context windows, or too many retrieved chunks may end up more expensive than a slightly pricier model that produces the same quality with less input. The same logic applies to model tiering. Running every task through a premium reasoning model often wastes budget when cheaper models can handle classification, extraction, moderation, or simple summaries. The best way to compare AI models is to measure how much it costs each one to complete a real workflow at the quality level you need. That number is more useful than the sticker price on a model card.

Conclusion

Cost per token tells you how an LLM is billed, but not what an AI feature will actually cost to run. Real spend depends on prompt length, output length, retrieval, caching, retries, and how many model calls sit behind one user action.

That is why the better pricing question is not “Which model is cheapest per million tokens?” It is “Which model completes this workflow at the lowest total cost for the quality I need?” If you want a realistic AI budget, measure cost per completed task, not token price in isolation.

Frequently Asked Questions

1. What is cost per token in AI?

It is the amount a provider charges for the tokens an AI model processes and generates. Most API pricing is based on input tokens and output tokens.

2. How much is 1 million tokens?

The answer depends on the model. Low-cost models may charge a few cents to a few dollars per 1 million input tokens, while premium models may charge much more, especially for output tokens.

3. Why are output tokens often more expensive?

Many providers price output higher because generation is more computationally expensive and because long responses increase serving cost.

4. What are cached input tokens?

Cached input tokens are repeated prompt prefixes or reused context that a provider stores and bills at a lower rate than standard input tokens.

5. Is token based pricing the same across all providers?

No. Providers differ in model tiers, output pricing, caching rules, reasoning behavior, tool charges, and multimodal billing.

6. Does RAG increase LLM cost?

Yes. Retrieval-augmented generation often increases input tokens because document chunks are inserted into the prompt.

7. Why is my AI bill higher than expected?

Common reasons include long prompts, growing chat history, long outputs, retries, agent loops, premium model use, and missing prompt caching.

8. Is a cheaper model always cheaper to run?

No. A cheaper listed model may produce longer outputs, use more reasoning tokens, or require more retries, which can increase total task cost.

9. How do I calculate cost per request?

Use this formula: (input tokens ÷ 1,000,000 × input price) + (output tokens ÷ 1,000,000 × output price)

10. Should I choose a model based only on token price?

No. You should compare quality, latency, context needs, output length, workflow complexity, and effective cost per completed task.