Model Evaluation Metrics — Tutorial

Why accuracy misleads on imbalanced data

A fraud detection model that predicts 'not fraud' for every transaction achieves 99.9% accuracy if fraud is 0.1% of transactions. That model is useless.

Accuracy = (TP + TN) / total. It works only when classes are balanced. For imbalanced problems — fraud, disease diagnosis, rare events — you need precision, recall, and F1.

Precision, recall, F1, and when to use each

Precision = TP / (TP + FP): of everything predicted positive, how many were actually positive? Use when false positives are costly (spam filter — don't block real emails).

Recall = TP / (TP + FN): of all actual positives, how many did we catch? Use when false negatives are costly (cancer screening — don't miss cases).

F1 = 2 · (precision · recall) / (precision + recall): harmonic mean — good overall metric when both matter.

ROC AUC: area under the ROC curve. Threshold-independent; measures how well the model separates classes. AUC=1 is perfect; AUC=0.5 is random.

Regression metrics

MAE (Mean Absolute Error): average |ŷ − y|. Same units as target. Robust to outliers. Easy to explain.

MSE (Mean Squared Error): average (ŷ − y)². Penalises large errors more. Differentiable — used as a training loss.

RMSE (Root MSE): √MSE. Same units as target. Most commonly reported.

R² (coefficient of determination): fraction of variance explained. R²=1 is perfect; R²=0 means the model is no better than predicting the mean. Can be negative for very bad models.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Model Evaluation Metrics — Tutorial.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path