LLM inference cost explained: API versus own GPUs

LLM inference cost is what an organisation pays to get answers from a language model. An API bills tokens times the price per token. On own GPUs, the cost is server or rental plus people, power and idle capacity, divided by the tokens served. Volume and the choice of API decide which is cheaper.

The break-even calculator applies these figures to an organisation's own volume. The guide to self-hosted LLMs covers the wider decision.

What is LLM inference cost?

Training teaches a model once; few organisations do it for large models. Inference is every use after that, and almost every organisation pays for it monthly.

Inference is measured in tokens: fragments of text, often a word or part of one. Input is the question, instructions and documents; output is the answer. Search and summarisation are input-heavy. Drafting and code generation are closer to balanced.

Which components make up the bill?

An API bundles hardware, power and operations into a price per token. Own GPUs turn the same costs into a fixed monthly bill, busy or idle.

Sources: Lindstead cost framework; pricing structure from the API providers' pricing pages; throughput factors from the vLLM documentation; accessed 28 September 2026.

How is API inference priced?

APIs charge per million tokens. Output costs more than input, because the model generates it one token at a time. At the ECB reference rate of 2 October 2026, Anthropic lists Claude Sonnet 5 at €1.78 per million input tokens and €8.91 per million output tokens. OpenAI lists GPT-6 Astra at €8.91 and €44.54.12 Hosted open-weight models cost far less: DeepSeek V4-Pro €1.18 and €3.53 at peak, half that off-peak; Mistral Large 3 €0.45 and €1.34.34 Most providers discount cached, repeated input.

Lindstead blends the two prices at three input tokens per output token, typical of document and search work. Per million tokens, that gives €3.56 for Claude Sonnet 5, €17.82 for GPT-6 Astra, €1.76 for DeepSeek V4-Pro at peak and €0.67 for Mistral Large 3. Other mixes need their own ratio.

How fast are API prices falling?

Quickly. Epoch AI found the price of a given performance level fell by between 9 and 900 times per year, depending on task and level, and cautioned that the fastest declines may not continue.5 Locking in today's API prices for three years flatters own infrastructure. GPU rental rose in 2026: Nebius raises on-demand prices on 1 October 2026, and the median H200 rental rose about a quarter in a year, as the current GPU rental prices in Europe show.10

What does it cost to run an LLM on your own GPUs?

Self-hosting swaps a price per token for four blocks of cost. Only hardware appears on a price list.

Hardware: buy or rent

An owned server is bought once and written off. A Mercatus vendor analysis puts an eight-GPU B200 server at about €20.22 per server-hour at 70 percent utilisation over three years, roughly €14,788 a month. That covers purchase net of resale value, GPU power, colocation and maintenance, not staff.6 Renting avoids the purchase. Finnish provider Verda lists an RTX PRO 6000 with 96 GB at €1.75 per GPU-hour, about €1,275 for a full month, and an H200 at €4.13.7 Model size decides the GPU; the guide to hardware needed per model size sets this out.

Throughput: tokens per GPU-hour

Hardware becomes a cost per token only when divided by the tokens it produces. That depends on the model, numeric precision, prompt and answer length, and how many requests run together. Serving software such as vLLM raises throughput by batching requests and managing memory per conversation.8 Only a measurement on the actual model and workload gives the figure.

Utilisation: idle capacity is paid for

An API charges only for tokens used. A server sized for the morning peak costs the same idling at night. Utilisation most often decides the case, which is why Mercatus states its 70 percent assumption.

People, power and facilities

Owned hardware needs rack space, power, cooling, network capacity and someone on call. Rented GPUs move facility costs to the provider, but engineers still run the inference server, model updates, access control and monitoring. No list price includes this, so the model states it as an assumption.

How does an API bill compare with your own infrastructure?

Break-even volume is the monthly cost of self-hosting divided by the API price per token. Above it, the own server is cheaper, if it can carry the load. A worked example with the published assumptions:

1. Volume. An organisation runs 2 billion tokens a month through a document assistant, three input tokens per output token.
2. API bill. At Claude Sonnet 5's blended €3.56 per million tokens, the bill is 2,000 × €3.56 = about €7,130 a month.
3. Own server. The owned eight-GPU B200 server costs about €14,790 a month, before staff.
4. Break-even. €14,790 divided by €3.56 per million tokens is about 4.2 billion tokens a month. At 2 billion, the API is cheaper.
5. Different API. Against GPT-6 Astra at €17.82, the same volume costs about €35,630 a month and break-even drops to about 0.8 billion tokens. The server wins, if an open model can do GPT-6 Astra's job.

Staff costs push both break-even points further out. The break-even calculator runs these steps on an organisation's own figures. For a decision, Lindstead prepares a three-year cost model with every assumption shown across API use, rented GPUs and owned servers.

Why is the choice of API the biggest variable?

Exhibit 1 shows a spread of more than twenty times between the cheapest and the most expensive API. Hardware prices vary far less. So break-even depends mostly on which API the server replaces, and whether the open model can do its job.

If an open model can replace a premium closed model, break-even volume is low. If a cheap hosted open-weight API would do, it is very high: a rented RTX PRO 6000 at about €1,270 a month only matches the Mistral Large 3 API at about 1.9 billion tokens a month. A cost-benefit study with authors from Carnegie Mellon University agrees: on-premise deployment breaks even within three months for small models, 6 to 24 months for medium ones and often over two years for large ones. It finds self-hosting viable mainly from about 50 million tokens a month, or under strict data residency rules.9

What does a cost model leave out?

• Quality and review. A cheaper model whose answers need more human checking is not cheaper.
• Price trends. API prices for a given capability keep falling; GPU prices rose in 2026.
• Growth. Doubled volume may need a second server; an API scales without a purchase decision.
• Control over data. Keeping prompts and documents in the organisation's own environment has no price in the model, yet often decides the case.

Data control weighs most in financial services, healthcare and legal and professional services, where client and patient data are strictly confidential. The comparison of on-premise and private cloud AI covers where such workloads can run; the Model Index lists open models and their licences.

Lindstead builds cost models across API use, rented GPUs and owned servers for an organisation's own workloads, and deploys the first use case on the chosen infrastructure. Schedule a meeting to review AI infrastructure costs.

Frequently asked questions

• How much does it cost to run an LLM per month?

  Through an API: tokens times the price per token, which varies widely by provider. On own GPUs: server or rental cost plus people, power and idle capacity, mostly independent of volume. The break-even calculator on lindstead.com shows both.

• Is self-hosting an LLM cheaper than an API?

  Only at high, steady volume, mainly when the open model replaces an expensive closed model. Against cheap hosted open-weight APIs, an owned eight-GPU server needs tens of billions of tokens a month to break even. Many organisations self-host for data control, not cost.

• What is a token and how is it priced?

  A fragment of text the model reads or writes, often a word or part of one. APIs charge per million tokens. Output costs more than input; cached, repeated input often costs less.

• How many tokens can one GPU serve?

  It depends on the model, numeric precision, prompt and answer length, and how many requests run together. Servers such as vLLM raise throughput by batching. Measure it on the actual workload before buying capacity.

Sources

1. Anthropic, Claude API pricing, accessed 28 September 2026. platform.claude.com/docs/en/about-claude/pricing
2. OpenAI, API pricing, accessed 28 September 2026. developers.openai.com/api/docs/pricing
3. DeepSeek, models and pricing, accessed 28 September 2026. api-docs.deepseek.com/quick_start/pricing
4. Mistral AI, API pricing, accessed 28 September 2026. mistral.ai/pricing/api
5. Epoch AI, LLM inference prices have fallen rapidly but unequally across tasks, 12 March 2025, accessed 28 September 2026. epoch.ai/data-insights/llm-inference-price-trends
6. Mercatus, B200 server price and cost per hour, 16 July 2026 (vendor analysis), accessed 28 September 2026. mercatus-ai.com/blog/b200-server-price
7. Verda, GPU pricing, accessed 28 September 2026. verda.com/pricing
8. Kwon et al., Efficient Memory Management for Large Language Model Serving with PagedAttention, SOSP 2023, accessed 28 September 2026. arxiv.org/abs/2309.06180; vLLM documentation, docs.vllm.ai
9. Pan, Wang et al., A Cost-Benefit Analysis of On-Premise Large Language Model Deployment: Breaking Even with Commercial LLM Services, arXiv 2509.18101, v3 11 November 2025, accessed 28 September 2026. arxiv.org/abs/2509.18101
10. Nebius, pricing including changes from 1 October 2026, and getdeploying.com, H200 price history, accessed 28 September 2026. nebius.com/prices