ModelRefs / Decoder-Only Architecture — AI Glossary
Decoder-Only Architecture — AI Glossary
A transformer that uses only the autoregressive decoder stack, generating tokens left-to-right conditioned on all prior tokens.
Overview
Decoder-only models (GPT, LLaMA, Claude, Gemini, Mistral) predict the next token given all preceding ones. Causal (left-to-right) masking prevents attending to future tokens. This architecture dominates large language models because it scales efficiently and handles both completion and instruction-following with a single objective.
Reference details
| Topic | architecture |
|---|---|
| Also known as | causal LM, autoregressive LM |
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Decoder-Only Architecture — AI Glossary.
Frequently asked questions
What is Decoder-Only Architecture?
A transformer that uses only the autoregressive decoder stack, generating tokens left-to-right conditioned on all prior tokens.
Is Decoder-Only Architecture the same as causal LM?
Yes — causal LM, autoregressive LM are common aliases for Decoder-Only Architecture.
What concepts are related to Decoder-Only Architecture?
Closely related concepts include encoder decoder, causal mask, transformer.