ModelRefs / PagedAttention — AI Glossary
PagedAttention — AI Glossary
A KV-cache memory manager that allocates attention memory in non-contiguous pages, eliminating fragmentation and enabling high-throughput serving.
Overview
Introduced in vLLM. Enables 2–4× higher throughput on the same GPU by packing many concurrent sequences. Inspired by OS virtual memory paging. Now standard in vLLM, SGLang, and most production inference engines.
Reference details
| Topic | inference |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Commonly confused with
Easily confused with FlashAttention because both are named after attention and both raise throughput, but they fix different problems. PagedAttention manages KV-cache *memory*, allocating it in pages so it does not fragment and requests can share prefixes. FlashAttention changes how attention is *computed*. They are complementary and serving stacks generally use both at once.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to PagedAttention — AI Glossary.
Frequently asked questions
What is PagedAttention?
A KV-cache memory manager that allocates attention memory in non-contiguous pages, eliminating fragmentation and enabling high-throughput serving.
What concepts are related to PagedAttention?
Closely related concepts include kv cache, vllm, throughput.