What is continuous learning
Continuous Learning studies the ability of a model to learn a series of tasks in sequence and retain the skills of the old tasks even after completing the new ones. It is closer to the human way of learning - constantly exposed to new data, rather than looking through all the data at once.
catastrophic forgetting
After training a neural network on an old task, if it continues to fine tune with data from a new task, the performance of the old task often drops sharply, a phenomenon known as "catastrophic forgetting". As early as 1989, McCloskey and Cohen documented this phenomenon.
Why do we learn the new and forget the old
The root cause lies in parameter sharing: weights in the same network serve multiple tasks simultaneously. When updating the weights of a new task, it will overwrite the values that the old task depends on, so the old knowledge is' flushed out '.
The three mainstream coping strategies
Regularization: Adding "protection" to important weights to restrict them from being modified by new tasks, represented by elastic weight consolidation (EWC, Kirkpatrick et al., 2017).
Replay/replay: When learning new data, mix in some old task samples or generate samples to remind the model not to forget.
Parameter isolation: Assign different subsets of parameters to different tasks or expand network capacity as needed.
Application and Boundary
Continuous learning is suitable for scenarios where data gradually arrives and it is inconvenient to store all historical data for a long time, such as long-term running recommendation systems, customer service robots, robot skill updates, etc. It does not conflict with pre training and fine-tuning: pre training provides a universal foundation, while continuous learning focuses on "steadily absorbing new knowledge without retraining the full amount". At present, it still faces the dilemma of balancing stability and plasticity.
Reference source
Comprehensive compilation of publicly available academic literature, including McCloskey and Cohen (1989), Kirkpatrick et al. (2017, PNAS, Elastic Weight Consolidation), etc.