Canary Rollout Duration Calculator
How long a new model version must run on a slice of traffic before you have seen enough requests to judge it — for a target request count, or to detect a given rize in the error rate with confidence.
A canary sees only its share of traffic, so the observations arrive slowly: at 5% of 200 requests a second the canary gets ten a second, and detecting a small change in a rare error needs a great many of them.
How the canary rollout duration calculator works
A canary sees only its share of traffic, so the observations arrive slowly: at 5% of 200 requests a second the canary gets ten a second, and detecting a small change in a rare error needs a great many of them. The requests needed to see a rize from p₀ to p₁ with the usual 95% confidence and 80% power come from the two-proportion formula, and the duration is simply that count over the canary's rate.
The result is often surprizing: a 1% error rate rizing to 1.5% needs tens of thousands of canary requests, which at a small traffic share is hours, not minutes.
Formula: n = (z_α √(2p̄(1−p̄)) + z_β √(p₀(1−p₀) + p₁(1−p₁)))² / (p₁ − p₀)²; duration = n / (RPS × share)
Worked examples
| Inputs | Canary duration to detect the rize | Note |
|---|---|---|
| 1% rizing to 1.5% at a 5% share | 12.9 minutes | hours, not minutes |
| A larger rize | 1.3 minutes | far fewer requests |
| A 50% split | 1.3 minutes | ten times faster than 5% |
FAQFrequently asked questions
What is a canary rollout?
Sending a small share of live traffic to a new version and watching its metrics before sending everyone. The canary is the early warning.
Why does it take so long?
Because errors are rare and the canary sees a small slice. Distinguishing a 1% error rate from 1.5% needs tens of thousands of requests, however you slice them.
What is statistical power?
The chance of detecting the rize if it is real. Eighty per cent is conventional; higher power needs more requests.
Should I use a bigger share?
If the blast radius is acceptable, yes — the duration falls in proportion. A 50% split judges ten times faster than 5%.
What about latency rather than errors?
Continuous metrics need far fewer samples than rare events. Use a t-test on latency percentiles; this page is for error rates and other proportions.
Where these figures come from
- Vaswani et al. (2017) — Attention Is All You Need — the transformer architecture the memory arithmetic follows
- Kaplan et al. (2020) — Scaling Laws for Neural Language Models — the compute relationship used for training estimates
- Hoffmann et al. (2022) — Training Compute-Optimal Large Language Models — the tokens-per-parameter guidance ("Chinchilla")
- IEEE 754 — Standard for Floating-Point Arithmetic — the numeric formats behind bytes per parameter
- NIST AI Risk Management Framework — the US federal AI framework
Last checked: September 2026. The relationships here are architectural, not vendor-specific: bytes per parameter follow the numeric format, KV-cache size follows the transformer definition, and token-per-word ratios come from published tokeniser behaviour.