ModelRefs / GRPO (Group Relative Policy Optimization) — AI Glossary
GRPO (Group Relative Policy Optimization) — AI Glossary
A reinforcement learning algorithm for LLMs that replaces the critic model with group-based advantage estimation, used in DeepSeek-R1.
Overview
GRPO (Shao et al. 2024) generates a group of completions per prompt, uses their relative rewards to estimate advantages (no separate critic model needed), and applies PPO-style clipping. Simpler and more compute-efficient than PPO for LLM fine-tuning. Used in DeepSeek-R1 and Qwen reasoning models to achieve o1-level math performance.
Reference details
| Topic | training |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Primary source
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to GRPO (Group Relative Policy Optimization) — AI Glossary.
Frequently asked questions
What is GRPO (Group Relative Policy Optimization)?
A reinforcement learning algorithm for LLMs that replaces the critic model with group-based advantage estimation, used in DeepSeek-R1.
What concepts are related to GRPO (Group Relative Policy Optimization)?
Closely related concepts include ppo, rlhf, verifiable reward.