LLM & RAG Running Cost Calculator
Model the monthly running cost of an LLM assistant or RAG application: tokens per conversation, retrieved context, caching, embeddings, re-indexing and vector database. Enter your own provider's prices, because model pricing changes often.
Estimate monthly LLM + RAG cost
How we calculate
Everything is calculated in your browser from the inputs above. Nothing is sent anywhere or stored. The formula has four parts:
- Messages = conversations × messages per conversation.
- Input tokens per message = prompt tokens + (retrieved chunks × tokens per chunk). Input cost = messages × input tokens × [(1 − cache hit rate) × input price + cache hit rate × input price × cached price %] ÷ 1,000,000.
- Output cost = messages × output tokens × output price ÷ 1,000,000. LLM token cost = (input cost + output cost) × (1 + overhead %).
- Embedding cost = (documents re-indexed × tokens per document + messages × query embedding tokens) × embedding price ÷ 1,000,000. Total = LLM token cost + embedding cost + vector database.
To turn that central figure into a range, we multiply the usage-based costs by 0.8 for the low end and 1.3 for the high end. The vector database fee stays fixed. Real conversations vary a lot, and long conversations with many retrieved chunks push costs up more than short ones pull them down, so the range is skewed upwards. Cost per conversation is the total range divided by conversations. One-off initial ingestion (documents × tokens per document × embedding price) is shown separately, with a range of 0.9–1.2× to allow for chunk overlap.
The default prices are neutral example values, not any provider's list price. Per-token pricing differs a lot between models and changes often, so always replace the examples with current rates from your provider.
Choosing realistic inputs
| Input | What drives it | How to measure it |
|---|---|---|
| Prompt tokens | System instructions, tool definitions, and conversation history that grows with each turn | Log token counts from your provider's API response during a pilot |
| Chunks × tokens per chunk | Your chunking strategy and top-k retrieval setting; re-ranking can cut the number of chunks sent | Check the retriever config; typical chunks are a few hundred tokens |
| Output tokens | Answer length, structured output, and reasoning tokens on models that bill for them | Average from pilot logs, not from the maximum token setting |
| Cache hit rate | A stable system prompt and tool list at the start of the prompt improves prefix caching | Cached-token fields in API usage data, where your provider reports them |
| Overhead | Retries, guardrail or classifier calls, query rewriting, agent tool loops, evaluation runs | Compare total billed tokens to user-facing calls for a week |
Agents that call tools often make several model calls per user message. If yours plans, calls a tool and then summarises, count that as extra messages or raise the overhead. Our production RAG architecture guide explains how ingestion, retrieval, re-ranking and evaluation affect these numbers.
Where running costs usually hide
- Retrieved context is usually the biggest input cost. Five 400-token chunks is 2,000 tokens on every message. Better re-ranking that sends three strong chunks instead of eight weak ones often saves more than switching models.
- Conversation history grows. Without summarisation or trimming, turn ten sends everything from turns one to nine.
- Output is priced higher. Output tokens often cost several times more than input tokens, so concise answer formats matter.
- Re-indexing whole corpora. Re-embedding only changed documents keeps ingestion costs trivial. Re-embedding everything nightly does not.
- Not every request needs the most capable model. Routing simple questions to a smaller model can cut the blended price per token.
Token cost is only part of the total cost of ownership. Engineering, evaluation, monitoring and support usually cost more in year one. Our enterprise LLM app TCO and ROI guide covers the full picture, and the AI agent cost calculator estimates build cost. If you want help designing a retrieval pipeline that stays cheap at scale, talk to our RAG development team.
Frequently asked questions
Why are the token prices editable instead of fixed?
Per-token prices differ widely between models and providers and change often. The defaults are neutral example values, so replace them with the current rates for the model you plan to use.
What usually makes RAG applications expensive to run?
Retrieved context sent on every message is usually the largest cost, followed by growing conversation history and long outputs. Retrieving fewer, better-ranked chunks and trimming history often saves more than switching models.
How does prompt caching affect cost?
When the start of a prompt, such as the system instructions and tool definitions, repeats across requests, some providers bill cached input tokens at a lower rate. Set the cache hit rate and the cached price percentage to match what your provider reports.
Does this include the cost of building the RAG system?
No. This calculator covers monthly running costs only. Engineering, evaluation, monitoring and support are separate; use the AI agent cost calculator for build estimates.
Want a second opinion on your token budget?
Copy your estimate and send it to us. We will review your retrieval design and model mix, then give you a fixed quote for building or optimising it.
Get a fixed quote