A model evaluation technique that divides data into multiple subsets, repeatedly training on some and testing on the remainder, to obtain a more reliable, unbiased estimate of model performance.
In Depth
Cross-validation solves a fundamental problem in model evaluation: if you evaluate a model on the same data used to train it, you get an optimistic, misleading picture of its real-world performance. The standard solution is to hold out a separate test set — but with limited data, dedicating a large portion to testing is expensive. Cross-validation uses all available data for both training and evaluation.
In k-fold cross-validation, the most common variant, the dataset is divided into k equal parts (folds). The model is trained k times — each time using k-1 folds for training and the remaining fold for evaluation. The k evaluation scores are averaged to produce a final performance estimate. With k=5 or k=10, this estimate is significantly more robust than a single train-test split, because every example is used for both training and testing across the k runs.
Cross-validation is especially important for Hyperparameter Tuning. By evaluating each hyperparameter configuration on cross-validation folds rather than a single test set, practitioners avoid 'overfitting to the test set' — a subtle error where repeated evaluation on the same test data inflates apparent performance. Leave-one-out cross-validation (LOOCV), where each data point is the test set once, is the extreme case — unbiased but computationally expensive for large datasets.
Cross-validation is how you honestly evaluate a machine learning model — providing a realistic performance estimate by ensuring every data point contributes to both training and testing across multiple runs.