ModelRefs / Grouped Query Attention (GQA) — AI Glossary
Grouped Query Attention (GQA) — AI Glossary
An attention variant sharing key-value heads across multiple query heads to reduce KV cache memory and inference cost. Also called GQA.
Overview
GQA (Ainslie et al. 2023) sits between multi-head attention (one KV per query head) and multi-query attention (one shared KV). It reduces KV cache memory proportionally to the group size while preserving most model quality. Used in LLaMA 3, Gemma, Mistral, and most modern open-weights models.
Reference details
| Topic | architecture |
|---|---|
| Also known as | GQA |
| Last reviewed | 2026-06-24 |
Related terms
Example: The KV cache is what this is buying
KV cache size is 2 (keys and values) × layers × KV heads × head dimension × sequence length × bytes per value. For 32 layers, 8 KV heads, head dimension 128, an 8,192-token sequence in fp16: 2 × 32 × 8 × 128 × 8,192 × 2 bytes = exactly 1 GiB — per concurrent sequence. Full multi-head attention with 32 KV heads would make that 4 GiB for the same request. Since the cache scales with context *and* concurrency, that factor of four is the difference between serving a handful of long-context users on a card and serving many. Quality barely moves; memory moves by the group ratio.
Commonly confused with
GQA sits between multi-head attention, where every query head has its own key-value pair, and multi-query attention, where all share one. It is a KV-cache optimisation, not a change to the quadratic attention computation — it does not make long context cheaper to compute, only cheaper to hold in memory.
When to use it
Reach for it when:
- Reasoning about serving capacity, where the KV cache and not the weights is the limit
- Interpreting a model card: the ratio of query heads to KV heads predicts memory per sequence
- Choosing between models for long-context, high-concurrency workloads
Reach for something else when:
- Expecting faster prefill or reduced attention computation — that is a different problem
- Assuming the ratio is free: pushing towards a single shared group does cost quality
- Estimating total memory from weights alone, ignoring cache growth with concurrency
Primary source
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Grouped Query Attention (GQA) — AI Glossary.
Frequently asked questions
What is Grouped Query Attention (GQA)?
An attention variant sharing key-value heads across multiple query heads to reduce KV cache memory and inference cost.
Is Grouped Query Attention (GQA) the same as GQA?
Yes — GQA are common aliases for Grouped Query Attention (GQA).
What concepts are related to Grouped Query Attention (GQA)?
Closely related concepts include attention, kv cache, multi head attention.