ModelRefs / Disaggregated Serving — AI Glossary
Disaggregated Serving — AI Glossary
An inference architecture that separates prefill and decode onto different GPU pools to optimize each phase independently.
Overview
Prefill (compute-bound) and decode (memory-bandwidth-bound) have opposing resource profiles. Disaggregated serving runs them on separate GPU pools—prefill on compute-dense H100s, decode on memory-bandwidth-optimized hardware—transferring KV caches between pools. Used at scale by major providers; improves MFU and cost efficiency.
Reference details
| Topic | inference |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Disaggregated Serving — AI Glossary.
Frequently asked questions
What is Disaggregated Serving?
An inference architecture that separates prefill and decode onto different GPU pools to optimize each phase independently.
What concepts are related to Disaggregated Serving?
Closely related concepts include chunked prefill, kv cache, pipeline parallelism.