ModelRefs / Canary Deployment — AI Glossary
Canary Deployment — AI Glossary
A deployment strategy routing a small fraction of traffic to a new model version to detect regressions before full rollout.
Overview
Canary deployments for LLMs: route 5% of requests to the new model version, monitor quality metrics (LLM judge scores, thumbs up/down, task success rate) and latency/error rates, and gradually increase traffic if metrics hold. Prevents regressions from a model update from affecting all users. Supported by AI gateways (LiteLLM, PortKey) via weighted routing.
Reference details
| Topic | ecosystem |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Commonly confused with
Both this and blue-green reduce release risk, by opposite mechanisms. A canary shifts a small fraction of live traffic to the new version and needs per-request routing plus metrics good enough to detect a regression in that fraction. Blue-green switches everything at once. So a canary limits blast radius but exposes some users; blue-green exposes none until it exposes all.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Canary Deployment — AI Glossary.
Frequently asked questions
What is Canary Deployment?
A deployment strategy routing a small fraction of traffic to a new model version to detect regressions before full rollout.
What concepts are related to Canary Deployment?
Closely related concepts include blue green deployment, a b testing llm, model routing.