ModelRefs / Fine-Tuning LLMs — Tutorial
Fine-Tuning LLMs — Tutorial
Supervised fine-tuning, instruction tuning, and when full fine-tuning beats prompting. Covers What fine-tuning actually changes.
Overview
Supervised fine-tuning, instruction tuning, and when full fine-tuning beats prompting
Level: Advanced. Estimated reading time: 45 minutes.
What fine-tuning actually changes
Fine-tuning updates the model's weights on a new dataset using gradient descent — the same process as pretraining, on a smaller task-specific corpus. Three main types:
Full fine-tuning: update all parameters. Most flexible but memory-intensive — a 7B model needs ~56GB GPU RAM in fp32.
Instruction fine-tuning (SFT): train on (instruction, response) pairs in a standardised chat format. This turns base models (next-token predictors) into assistants.
Task-specific fine-tuning: train on (input, expected_output) pairs for a narrow task — SQL generation, medical report extraction, code review. The model learns the task's conventions and output format.
Data requirements and formatting
Quality >> quantity. A few hundred high-quality examples often outperform thousands of noisy ones.
Practical starting points: - Simple classification/extraction: 100–500 examples - Instruction following on a specific domain: 1,000–5,000 examples - Style/persona alignment: 500–2,000 examples
OpenAI/Hugging Face format: {"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
Loss is computed only on the assistant response tokens — the model learns to produce the desired output, not to predict the instruction itself.
Data hygiene: deduplicate, remove PII, verify outputs are correct. A test split (10–20%) is essential for early stopping.
Training recipe and hyperparameters
Standard fine-tuning hyperparameters that work for most 7B–13B models:
Learning rate: 1e-5 to 3e-5 (much lower than pretraining's 3e-4). Use cosine decay with a 3–5% warmup.
Batch size: 8–32 (with gradient accumulation to simulate larger batches).
Epochs: 1–3. More epochs on a small dataset causes overfitting — the model memorises training examples. Monitor validation loss.
Sequence length: 512–2048 tokens. Truncate or filter training examples that exceed your maximum.
The TRL library from Hugging Face provides SFTTrainer, which handles tokenisation, masking prompt tokens from loss, and packing short examples to fill the context window.
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Fine-Tuning LLMs — Tutorial.