ModelRefs / Encoder-Only Architecture — AI Glossary
Encoder-Only Architecture — AI Glossary
A transformer encoder producing rich bidirectional representations of input text, used for classification, retrieval, and NER. Also called BERT-style model.
Overview
Encoder-only models (BERT, RoBERTa, DeBERTa) see the full input simultaneously with bidirectional attention, producing contextual embeddings rather than generating text. Ideal for semantic search, sentence similarity, text classification, and token labeling. Typically much smaller than generative LLMs.
Reference details
| Topic | architecture |
|---|---|
| Also known as | BERT-style model |
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Encoder-Only Architecture — AI Glossary.
Frequently asked questions
What is Encoder-Only Architecture?
A transformer encoder producing rich bidirectional representations of input text, used for classification, retrieval, and NER.
Is Encoder-Only Architecture the same as BERT-style model?
Yes — BERT-style model are common aliases for Encoder-Only Architecture.
What concepts are related to Encoder-Only Architecture?
Closely related concepts include encoder decoder, embedding model, bi encoder.