Inference Energy Calculator
The electricity an AI service uses per query and per million tokens from the accelerator’s power draw and its throughput — with the cost at your tariff and the emissions on your grid.
A GPU drawing 700 W that serves 2,000 tokens a second uses 0.
How the inference energy calculator works
A GPU drawing 700 W that serves 2,000 tokens a second uses 0.35 joules per token — 0.1 Wh per thousand, about 97 Wh per million tokens. Add the overhead of the rest of the server and the data centre (power usage effectiveness, typically 1.1–1.5), multiply by your tariff for cost and by the grid factor for CO₂. The per-query row uses your average tokens per response.
Formula: Wh per token = power × PUE ÷ (throughput × 3,600); cost = kWh × tariff; CO₂ = kWh × grid factor
Worked examples
| Inputs | Energy per million tokens | Note |
|---|---|---|
| 700 W, 2,000 tok/s, PUE 1.3 | 0.126 kWh | 0.126 kWh per million tokens |
| A large model at 300 tok/s | 0.843 kWh | 0.84 kWh per million |
| An edge server at 50 W and 100 tok/s | 0.139 kWh | 0.139 kWh per million |
FAQFrequently asked questions
Is this the whole energy cost of AI?
No — it is serving only. Training is a one-off cost spread over every query the model ever answers, and embodied energy in the hardware adds more. For a widely used model, serving dominates over its life.
How does this compare with a web search?
A typical estimate for a search is about 0.3 Wh; a 500-token chatbot answer on the defaults here is under 0.1 Wh at the accelerator and 0.13 with overhead — smaller than headlines suggest, though long contexts and large models raise it several-fold.
What throughput should I use?
Aggregate tokens per second across all concurrent requests on the accelerator, not one user’s stream. Batched serving of a mid-size model reaches thousands of tokens a second on a data-centre GPU.
Why include PUE?
The GPU is not alone: CPUs, memory, networking, cooling and power conversion add 10–50% on top. Hyperscale data centres report PUE near 1.1; enterprise rooms 1.5 or more.
Where these figures come from
- Vaswani et al. (2017) — Attention Is All You Need — the transformer architecture the memory arithmetic follows
- Kaplan et al. (2020) — Scaling Laws for Neural Language Models — the compute relationship used for training estimates
- Hoffmann et al. (2022) — Training Compute-Optimal Large Language Models — the tokens-per-parameter guidance ("Chinchilla")
- IEEE 754 — Standard for Floating-Point Arithmetic — the numeric formats behind bytes per parameter
- National AI Centre — Australia's national AI body
Last checked: September 2026. The relationships here are architectural, not vendor-specific: bytes per parameter follow the numeric format, KV-cache size follows the transformer definition, and token-per-word ratios come from published tokeniser behaviour.