Fine-Tuning Cost Calculator
What fine-tuning costs — the run itself plus serving — and the request volume at which it beats putting the same examples in every prompt.
Fine-tuning trades a one-off training cost against a permanently shorter prompt.
How the fine-tuning cost calculator works
Fine-tuning trades a one-off training cost against a permanently shorter prompt. Few-shot prompting pays for those examples on every single request; a fine-tune pays once.
So there is a break-even volume, and it is usually lower than people expect. If a fine-tune removes 3,000 tokens from every prompt, it pays for itself in tens of thousands of requests — but if traffic is light, or the examples change often, prompting wins outright.
Formula: break-even = training cost / tokens saved per request × price
Worked examples
| Inputs | Training cost | Note |
|---|---|---|
| 5,000 examples, 3,000 tokens saved | 96 | pays back in weeks |
| Light traffic | 96 | prompting wins |
| A serving premium | 96 | halves the saving |
FAQFrequently asked questions
When is fine-tuning worth it?
When traffic is high enough that the prompt tokens it removes cost more than the training run. At scale that is usually within weeks.
What does fine-tuning actually save?
Prompt length, mainly — the few-shot examples you would otherwize send every request. It also tends to improve format adherence.
How many examples do I need?
Hundreds for format and tone; thousands for genuinely new behaviour. Quality matters far more than quantity.
Should I fine-tune or use retrieval?
Fine-tune for behaviour and format; retrieve for facts. Fine-tuning is a poor way to teach a model information that changes.
What is the hidden cost?
Maintenance. Every base-model upgrade means retraining, and a fine-tune ties you to a model version in a way prompting does not.
Where these figures come from
- Vaswani et al. (2017) — Attention Is All You Need — the transformer architecture the memory arithmetic follows
- Kaplan et al. (2020) — Scaling Laws for Neural Language Models — the compute relationship used for training estimates
- Hoffmann et al. (2022) — Training Compute-Optimal Large Language Models — the tokens-per-parameter guidance ("Chinchilla")
- IEEE 754 — Standard for Floating-Point Arithmetic — the numeric formats behind bytes per parameter
- NIST AI Risk Management Framework — the US federal AI framework
Last checked: September 2026. The relationships here are architectural, not vendor-specific: bytes per parameter follow the numeric format, KV-cache size follows the transformer definition, and token-per-word ratios come from published tokeniser behaviour.