ModelRefs / RLHF (Reinforcement Learning from Human Feedback) — AI Glossary
RLHF (Reinforcement Learning from Human Feedback) — AI Glossary
An alignment pipeline that trains a reward model from human preference comparisons and optimizes the LLM policy against it. Also called RLHF.
Overview
The classic RLHF pipeline (SFT → reward model → PPO) made ChatGPT possible. Modern stacks replace PPO with DPO or KTO for simplicity and stability. Human preference collection is the most expensive part.
Reference details
| Topic | training |
|---|---|
| Also known as | RLHF |
| Last reviewed | 2026-06-24 |
Related terms
Example: What the human actually provides
Annotators rarely write ideal answers. They see two model responses to the same prompt and pick the better one. Those comparisons train a reward model that scores responses, and the policy is then optimised against that score. The pipeline is built on preferences because ranking two options is far more reliable and cheaper than authoring a gold answer.
Commonly confused with
RLHF optimises a learned proxy for human preference, not human preference itself. Push hard enough and the policy exploits the reward model — the classic result being answers that are longer, more hedged and more agreeable because those correlate with approval in the training data.
When to use it
Reach for it when:
- You need broad behavioural shaping across open-ended tasks
- Quality is a judgement call rather than a checkable answer
- You can fund and sustain preference collection
Reach for something else when:
- Correctness is verifiable — train against the verifier instead
- You lack the annotation budget; a weak reward model is worse than none
- DPO or a similar direct method would achieve the same with far less machinery
Referenced by
This term is used by the following ModelRefs references:
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to RLHF (Reinforcement Learning from Human Feedback) — AI Glossary.
Frequently asked questions
What is RLHF (Reinforcement Learning from Human Feedback)?
An alignment pipeline that trains a reward model from human preference comparisons and optimizes the LLM policy against it.
Is RLHF (Reinforcement Learning from Human Feedback) the same as RLHF?
Yes — RLHF are common aliases for RLHF (Reinforcement Learning from Human Feedback).
What concepts are related to RLHF (Reinforcement Learning from Human Feedback)?
Closely related concepts include sft, dpo, reward model, alignment.