论文标题
对数持续学习
Logarithmic Continual Learning
论文作者
论文摘要
我们介绍了一种神经网络结构，该神经网络体系结构从对数中减少了不断学习模型的生成彩排中的自我外行步骤的数量。在持续学习（CL）中，培训样本是在随后的任务中进行的，训练有素的模型一次只能访问一个任务。为了重播先前的样本，现代CL方法引导生成模型，并通过当前和再生的过去数据递归训练它们。这种复发导致了多余的计算，因为在每个任务后都会重新生成相同的过去样本，并且重建质量依次降低。在这项工作中，我们解决了这些局限性，并提出了一种新的生成彩排架构，该结构需要每个样本的大多数对数次数的再培训。我们的方法利用了一组生成模型中的过去数据分配，因此大多数人不需要在任务后重新培训。我们对数持续学习方法的实验评估表明，我们方法相对于最先进的生成彩排方法的优越性。
We introduce a neural network architecture that logarithmically reduces the number of self-rehearsal steps in the generative rehearsal of continually learned models. In continual learning (CL), training samples come in subsequent tasks, and the trained model can access only a single task at a time. To replay previous samples, contemporary CL methods bootstrap generative models and train them recursively with a combination of current and regenerated past data. This recurrence leads to superfluous computations as the same past samples are regenerated after each task, and the reconstruction quality successively degrades. In this work, we address these limitations and propose a new generative rehearsal architecture that requires at most logarithmic number of retraining for each sample. Our approach leverages allocation of past data in a~set of generative models such that most of them do not require retraining after a~task. The experimental evaluation of our logarithmic continual learning approach shows the superiority of our method with respect to the state-of-the-art generative rehearsal methods.
