论文标题
有效估计培训实例的影响
Efficient Estimation of Influence of a Training Instance
论文作者
论文摘要
了解训练实例对神经网络模型的影响会提高可解释性。但是，评估影响很困难且效率低下，这表明如果未使用培训实例，模型的预测将如何更改。在本文中，我们提出了一种估计影响的有效方法。我们的方法的灵感来自辍学，它零遮盖子网并防止子网络学习每个培训实例。通过在辍学面罩之间切换，我们可以使用学习或不学习每个培训实例并估算其影响力的子网络。通过在分类数据集上对BERT和VGGNET进行的实验，我们证明了所提出的方法可以捕获训练的影响，增强错误预测的解释性，并清洁训练数据集以改善概括。
Understanding the influence of a training instance on a neural network model leads to improving interpretability. However, it is difficult and inefficient to evaluate the influence, which shows how a model's prediction would be changed if a training instance were not used. In this paper, we propose an efficient method for estimating the influence. Our method is inspired by dropout, which zero-masks a sub-network and prevents the sub-network from learning each training instance. By switching between dropout masks, we can use sub-networks that learned or did not learn each training instance and estimate its influence. Through experiments with BERT and VGGNet on classification datasets, we demonstrate that the proposed method can capture training influences, enhance the interpretability of error predictions, and cleanse the training dataset for improving generalization.
