Traditional deep learning models typically require thousands of annotated samples to learn a task well. But humans are different: just by looking at two or three photos of unfamiliar animals, you can recognize them. Can AI also achieve the ability to draw analogies? Few Shot Learning is the direction for studying this problem.
1、 What small sample learning needs to solve
In standard supervised learning, the more labeled samples a model sees, the better its performance is usually. In many real-world scenarios, annotated data is extremely scarce: rare disease images, new product categories, industrial defects... with few samples and expensive annotations, the model is prone to overfitting. The goal of small sample learning is to enable the model to complete classification or recognition tasks with only a small number of samples (such as 1-5 per class).
2、 Several key concepts
-N-way K-shot: In a task, give N categories and K samples for each category as the "support set", and then take new samples to determine which category it belongs to.
-Support set and query set: The support set is an example for the model to see, and the query set is a question for the model to answer.
-Meta Learning: Not directly learning a specific task, but learning the ability to "quickly learn new tasks", that is, "learn to learn".
3、 Common ideas
1. Based on metrics: Learn a feature space that brings similar samples closer and dissimilar samples farther apart, and then classify them using simple rules such as nearest neighbors (such as twin networks, prototype networks).
2. Based on optimization: Initialize the model to a parameter starting point that is easy to fine tune, and a few gradient updates can adapt to new tasks (such as MAML).
3. Based on external knowledge: Using prior knowledge from memory modules or pre trained large models, "prompt" new tasks to the model to complete.
4、 The relationship with the big model
Nowadays, big language models exhibit strong "contextual learning" abilities: by providing a few examples in prompts, they can complete new tasks accordingly. This is essentially a "few sample" ability that does not require parameter modification, and also transforms small sample learning from a "training technique" to a daily usable "usage technique".
5、 Application and Limitations
Small sample learning is commonly used in scenarios such as image classification, text classification, object recognition, and cold start recommendation. Its limitations are also evident: performance is highly dependent on task and data distribution, it remains unstable even with very few samples, and the results vary greatly depending on the evaluation protocol.
Summary in one sentence: Small sample learning attempts to free AI from the "data thirst" and quickly grasp new tasks from a small number of examples in a more human like way - it is both an old problem in machine learning and a new entry point in the era of big models.
[Reference source] Comprehensive compilation of industry information publicly released (such as Matching Networks, Prototypal Networks, MAML, and other public papers).