A modeling failure where a machine learning model learns the training data too closely — memorizing noise and edge cases — and subsequently performs poorly on new, unseen data.
In Depth
Overfitting is one of the most fundamental failure modes in Machine Learning. A model that overfits has learned the training data so precisely that it has essentially memorized it — including random noise, outliers, and idiosyncrasies specific to that particular dataset. When confronted with new data, the model performs poorly because its 'knowledge' doesn't generalize beyond the examples it has seen.
The intuition is like a student who memorizes every practice exam verbatim but can't solve any problem phrased differently. The model achieves high accuracy on training data — sometimes near 100% — while its accuracy on a held-out test set is significantly lower. This gap between training and test performance is the signature of overfitting. Its counterpart, underfitting, occurs when a model is too simple to capture even the underlying patterns in training data.
The antidote to overfitting is regularization — a family of techniques that constrain model complexity to force generalization. L1/L2 regularization penalizes large parameter values. Dropout randomly disables neurons during training. Early stopping halts training when test performance begins to degrade. Data augmentation artificially expands training data. Cross-validation helps detect overfitting before deployment. All of these address the same root cause: too much model complexity relative to the data available.
Overfitting is the model learning the map instead of the territory — a powerful but brittle system that fails whenever reality looks slightly different from its training examples.