ModelRefs / Hyperparameter — AI Glossary
Hyperparameter — AI Glossary
A configuration value set before training (learning rate, batch size, LoRA rank) that governs the training process but is not learned from data.
Overview
Hyperparameters control how a model learns, contrasted with parameters (weights) which are learned. Key LLM hyperparameters: learning rate, warmup steps, weight decay, batch size, context length, LoRA rank/alpha, and dropout rate. Hyperparameter tuning methods: grid search, random search, Bayesian optimization (W&B Sweeps, Optuna). Poor hyperparameter choices can prevent convergence or cause overfitting.
Reference details
| Topic | training |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Hyperparameter — AI Glossary.
Frequently asked questions
What is Hyperparameter?
A configuration value set before training (learning rate, batch size, LoRA rank) that governs the training process but is not learned from data.
What concepts are related to Hyperparameter?
Closely related concepts include experiment tracking, curriculum learning, lora.