ModelRefs / Cross-Attention — AI Glossary
Cross-Attention — AI Glossary
An attention mechanism where queries come from the decoder and keys/values come from the encoder, linking input and output.
Overview
Cross-attention is the bridge in encoder-decoder models: decoder tokens query encoded input representations. Also appears in multimodal transformers (image tokens attending to text queries) and retrieval-augmented generation (query attending to retrieved document representations).
Reference details
| Topic | architecture |
|---|---|
| Also known as | encoder-decoder attention |
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Cross-Attention — AI Glossary.
Frequently asked questions
What is Cross-Attention?
An attention mechanism where queries come from the decoder and keys/values come from the encoder, linking input and output.
Is Cross-Attention the same as encoder-decoder attention?
Yes — encoder-decoder attention are common aliases for Cross-Attention.
What concepts are related to Cross-Attention?
Closely related concepts include multi head attention, encoder decoder, multimodal architecture.