Zero-Shot & Few-Shot Learning

Zero-shot learning is a model's ability to perform a task it was never explicitly trained on, using only a natural language description. For example, a language model that was never specifically trained for sentiment analysis can classify movie reviews as positive or negative simply when prompted: 'Is this review positive or negative?' Few-shot learning extends this by providing a small number of examples in the prompt — showing the model two or three input-output pairs before asking it to handle a new case. This 'teaching by example' within the prompt is also called in-context learning.

These capabilities emerged as a surprising consequence of training large language models on diverse internet text. Because the training data contains examples of virtually every text format and task — reviews, translations, Q&A, code, etc. — large models internalize patterns that transfer across tasks. GPT-3 demonstrated that few-shot learning could achieve competitive results on benchmarks without any gradient updates (fine-tuning), a paradigm shift that made powerful AI accessible through prompting rather than machine learning engineering.

Zero-shot and few-shot learning have transformed how AI applications are built. Instead of collecting thousands of labeled examples and training a custom model for each new task (the traditional ML pipeline), developers can now solve many problems by crafting the right prompt with a few illustrative examples. This dramatically reduces development time and cost. However, performance is generally lower than fine-tuned models for complex or domain-specific tasks, and results are sensitive to the choice and ordering of examples. Understanding when to use few-shot prompting versus fine-tuning is a key practical skill.