Cost Per Correct Answer Calculator
Compare two models by the cost of a correct result rather than the cost of a call — price per task, accuracy on your evaluation, and the cost of retries or human review for the ones they get wrong.
A cheaper model is not cheaper if it is wrong more often and every wrong answer costs a retry or a review.
How the cost per correct answer calculator works
A cheaper model is not cheaper if it is wrong more often and every wrong answer costs a retry or a review. Divide each model's cost per task by its accuracy and you get the cost per correct answer; add the cost of handling its failures and you have the real comparison.
The break-even row says how accurate the cheaper model would need to be to win — often a few points above where it is.
Formula: cost per correct = cost per task / accuracy; effective = cost per task + (1 − accuracy) × failure cost
Worked examples
| Inputs | Cheaper per correct answer, all in | Note |
|---|---|---|
| A four-times cheaper model, nine points less accurate | Model B — 0.012 against 0.0165 per task | B still wins at this failure cost |
| Expensive failures | Model A — 0.057 against 0.093 per task | A wins |
| No cost to failures | Model B — 0.003 against 0.012 per task | B wins easily |
FAQFrequently asked questions
Why cost per correct answer?
Because a call that produces a wrong answer bought nothing. Dividing by accuracy prices the output you actually wanted, and makes a cheap inaccurate model comparable with a dear accurate one.
What is the failure cost?
Whatever a wrong answer costs you: a retry with a bigger model, a human review, a refund, a lost customer. It is the number that decides most comparisons, so estimate it honestly.
How do I measure accuracy?
On your own evaluation set, not a public benchmark — a few hundred representative tasks with known answers. Benchmarks rarely match your distribution.
What about latency and quality beyond right or wrong?
This page treats output as correct or not. For graded quality, use a score in place of accuracy; for latency, the response-time page.
Should I route between models?
Often — send easy tasks to the cheap model and hard ones to the expensive one. The break-even accuracy row tells you how good the cheap model has to be on the tasks you send it.
Where these figures come from
- Vaswani et al. (2017) — Attention Is All You Need — the transformer architecture the memory arithmetic follows
- Kaplan et al. (2020) — Scaling Laws for Neural Language Models — the compute relationship used for training estimates
- Hoffmann et al. (2022) — Training Compute-Optimal Large Language Models — the tokens-per-parameter guidance ("Chinchilla")
- IEEE 754 — Standard for Floating-Point Arithmetic — the numeric formats behind bytes per parameter
- AI Safety Institute — the UK government AI body
Last checked: September 2026. The relationships here are architectural, not vendor-specific: bytes per parameter follow the numeric format, KV-cache size follows the transformer definition, and token-per-word ratios come from published tokeniser behaviour.