论文标题
无任务持续学习的神经Dirichlet过程混合模型
A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
论文作者
论文摘要
尽管对持续学习的兴趣日益增加，但在一个相当有限的设置中，已经研究了大多数当代作品，在这个设置中，任务明显区分，并且在培训期间已知任务边界。但是，如果我们的目标是开发一种像人类一样学习的算法，那么这种设置远非现实，并且必须开发以无任务方式工作的方法。同时，在持续学习的几个分支中，基于扩展的方法具有消除灾难性遗忘的优势，通过分配新资源来学习新数据。在这项工作中，我们提出了一种基于扩展的无任务持续学习方法。我们的模型被称为连续的神经差异过程混合物（CN-DPM），由一组负责数据子集的神经网络专家组成。 CN-DPM在贝叶斯非参数框架下以原则上的方式扩大了专家的数量。通过广泛的实验，我们表明我们的模型成功地执行了歧视和生成任务（例如图像分类和图像生成）的无任务持续学习。
Despite the growing interest in continual learning, most of its contemporary works have been studied in a rather restricted setting where tasks are clearly distinguishable, and task boundaries are known during training. However, if our goal is to develop an algorithm that learns as humans do, this setting is far from realistic, and it is essential to develop a methodology that works in a task-free manner. Meanwhile, among several branches of continual learning, expansion-based methods have the advantage of eliminating catastrophic forgetting by allocating new resources to learn new data. In this work, we propose an expansion-based approach for task-free continual learning. Our model, named Continual Neural Dirichlet Process Mixture (CN-DPM), consists of a set of neural network experts that are in charge of a subset of the data. CN-DPM expands the number of experts in a principled way under the Bayesian nonparametric framework. With extensive experiments, we show that our model successfully performs task-free continual learning for both discriminative and generative tasks such as image classification and image generation.
