Cross-Validation — Tutorial

Why a single train-test split isn't enough

A single 80/20 split may give an optimistic or pessimistic accuracy estimate depending on which 20% ended up in the test set. If your dataset has 500 samples, the 100-sample test set is noisy — a different random split can shift accuracy by ±3%.

Cross-validation (CV) repeatedly splits the data, trains, and evaluates, then averages the scores. It gives a more stable estimate of generalisation performance and uses all data for both training and testing (at different times).

K-fold: the standard approach

K-fold CV divides data into K equally sized folds. Each fold takes a turn as the test set while the other K−1 folds train the model. Final score = mean of K test scores.

Typical values: K=5 or K=10. K=5 is faster; K=10 gives lower variance estimates. K=n (leave-one-out) is exact but expensive on large datasets.

Stratified K-fold preserves class proportions in each fold — always use this for classification on imbalanced data.

Nested CV for unbiased hyperparameter tuning

When you tune hyperparameters using CV, the chosen hyperparameters are optimised for that validation set — the CV score is slightly optimistic. For an unbiased estimate of the tuned model's generalisation performance, use nested CV: an outer loop estimates performance, an inner loop tunes hyperparameters.

In practice: use GridSearchCV or RandomizedSearchCV (which do inner CV), then report their best score from an outer cross_val_score. This is the correct protocol for papers and benchmarks.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Cross-Validation — Tutorial.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path