What Will This LLM Actually Cost to Run?
Token pricing looks trivially cheap per thousand and becomes a significant line item at organisational scale. This calculator turns usage assumptions into a monthly figure, a per-user cost and an annual run rate, which is what a budget conversation actually needs.
Model one application. The per-query and per-user figures are usually more useful for decision-making than the monthly total, because they scale predictably.
LLM Cost Calculator
Replace the default token prices with current rates for your chosen model.
Question plus retrieved context plus system prompt.
Check the provider’s current published rate.
Vector database, hosting, monitoring, embeddings.
Tokens plus supporting infrastructure. Excludes build and staff time.
Indicative only. Token prices vary by provider and model tier and change frequently — replace the defaults with current published rates. Build cost, engineering time and ongoing maintenance are not included.
Reading the Numbers
LLM costs behave differently from conventional software costs. They scale with usage rather than with seats, which changes how you budget and where the risks sit.
Output tokens dominate the bill
Output is typically priced several times higher than input on most commercial models. An application producing long responses will cost far more than one producing short ones at identical query volume, which makes response length a genuine cost lever rather than only a UX decision.
Retrieval context inflates input rapidly
Retrieval-augmented applications send retrieved documents with every single query. Ten thousand tokens of context per query is common and utterly dwarfs the user’s actual question, which is why context size is usually the first thing to optimise.
Cost per query is the number to track
The monthly total moves with adoption and tells you little on its own. Cost per query is stable, comparable across models, and immediately reveals whether a change made things better or worse. It is the metric worth putting on a dashboard.
How the Estimate Is Built
Straightforward arithmetic over your usage assumptions. Every input is one you can measure once the application is running, and estimate reasonably before it is.
Monthly query volume
Active users multiplied by queries per user per working day, across twenty-two working days a month.
Tokens per query
Input tokens including any retrieved context and system prompt, plus expected output tokens. Roughly 750 words per thousand tokens.
Priced separately by direction
Input and output tokens are multiplied by their respective per-million rates, since output is almost always the more expensive of the two.
Infrastructure added on top
Vector database, hosting, monitoring and supporting services are added as a flat monthly cost to give a realistic total.
The Four Levers That Actually Reduce Cost
In roughly the order most organisations should try them. The first two are usually worth more than switching models.
Shrink the context before you change the model
Most retrieval applications send far more context than the answer requires, usually because the retrieval step was tuned for recall rather than precision. Returning three highly relevant chunks instead of ten mediocre ones frequently improves answer quality and reduces input cost at the same time.
- Tune retrieval for precision rather than maximum recall
- Re-rank results and pass only the top few to the model
- Trim boilerplate and repetition from system prompts
- Measure the quality impact — smaller context often scores better
Route by difficulty rather than using one model for everything
Most workloads contain a large proportion of straightforward queries that a smaller, cheaper model handles perfectly well, and a minority that genuinely need a frontier model. Routing by complexity commonly cuts spend substantially with no perceptible quality change.
- Classify queries by complexity and route accordingly
- Use smaller models for extraction, classification and summarisation
- Reserve the most capable model for genuinely difficult reasoning
- Measure quality per tier so routing decisions are evidence-based
Cache what repeats
Organisational usage is far more repetitive than people expect. The same policy questions, the same lookups, the same document summaries recur constantly. Caching frequent responses and using provider-side prompt caching for stable system prompts both remove cost without any quality trade-off.
- Cache responses to frequently repeated queries
- Use provider prompt caching for long, stable system prompts
- Cache retrieval results for common questions
- Track cache hit rate as a first-class metric
Consider self-hosting only at genuine scale
Self-hosted open-weight models can be cheaper per token but carry fixed infrastructure and engineering costs that only amortise at high, sustained volume. Below that threshold, API pricing almost always wins once staff time is counted honestly.
- Compare total cost of ownership, not just per-token pricing
- Include GPU infrastructure, engineering time and ongoing maintenance
- Remember idle capacity is paid for whether used or not
- Data sovereignty may justify self-hosting independently of cost
Next Steps
Build vs Buy Calculator
Compare a custom build against per-seat commercial licences over three years.
Compare the options →RAG vs Fine-Tuning Decision Tool
Architecture drives cost more than model choice. Work out which approach fits.
Choose an approach →Private LLM Cost in Australia
The longer written guide to what private deployments cost locally.
Read the guide →Frequently Asked Questions
A thousand tokens is roughly 750 words of English, so estimate in words and convert. For a typical internal knowledge assistant, budget 200 to 500 tokens for the user question, 2,000 to 10,000 for retrieved context depending on how many document chunks you pass, 200 to 800 for the system prompt, and 300 to 800 for the response. That yields something in the range of 3,000 to 12,000 input tokens and 300 to 800 output tokens per query. Once you have a prototype, measure rather than estimate — real usage almost always differs from the assumption, usually upward on context.
It reflects how the models actually work. Input tokens are processed in parallel in a single forward pass, while output tokens are generated sequentially, each one requiring a full pass through the model conditioned on everything before it. Generating a thousand tokens is therefore far more computationally expensive than reading a thousand. The practical consequence is that response length is a real cost lever: instructing a model to be concise reduces spend meaningfully, and is often better for users anyway.
For a retrieval-augmented application, the usual additions are a vector database, application hosting, embedding generation for your document corpus, monitoring and observability tooling, and any document processing pipeline. For a small internal deployment this commonly runs a few hundred dollars a month; for a large corpus with frequent re-indexing it can be considerably more. The cost people most often forget is re-embedding — every time you change embedding model or reprocess the corpus, you pay for the whole corpus again.
The defaults are indicative placeholders rather than live pricing, and you should replace them with the current published rates for whichever model you actually intend to use. Provider pricing changes frequently, generally downward, and varies substantially between model tiers — often by an order of magnitude or more between a small model and a frontier one. Always check the provider’s current pricing page before relying on any figure here for a budget submission.
Later than most teams assume, and the honest answer depends heavily on your utilisation. GPU infrastructure capable of serving a reasonable open-weight model at production latency costs a substantial fixed monthly amount whether you use it or not, and you also pay for the engineering time to deploy, monitor, update and troubleshoot it. Sustained high utilisation is what makes the economics work. Below that, API pricing usually wins once staff time is counted honestly. Data sovereignty, latency and regulatory requirements are frequently better reasons to self-host than cost is.
Yes, and this catches teams out more than any pricing detail. Unlike per-seat software, LLM cost scales with usage, so a successful pilot that triples in adoption triples the bill. Budget against the per-user figure rather than the current monthly total, set spending alerts at the provider level from day one, and consider per-user rate limits for internal applications. The failure mode to avoid is an enthusiastic rollout with no cost controls, which is a genuinely uncomfortable conversation with finance a month later.
Want This Modelled Properly?
Send us your use case and expected volumes and we will model the real cost across architectures — including telling you when a smaller model or a simpler approach would do the job.