论文标题
朝着强大的K-Nearest-Neighbor机器翻译
Towards Robust k-Nearest-Neighbor Machine Translation
论文作者
论文摘要
K-Nearest-Neighbor机器翻译（KNN-MT）近年来成为NMT的重要研究方向。它的主要思想是从附加数据存储中检索有用的键值对，以在不更新NMT模型的情况下修改翻译。但是，基础检索到的嘈杂对将极大地恶化模型性能。在本文中，我们进行了一项初步研究，发现该问题并未完全利用NMT模型的预测。为了减轻噪音的影响，我们通过强大的训练提出了一种富有信心的KNN-MT模型。具体而言，我们引入了NMT置信度，以完善KNN-MT的两个重要组成部分的建模：KNN分布和插值。同时，我们将两种类型的扰动注入检索对以进行健壮的训练。四个基准数据集的实验结果表明，我们的模型不仅比当前的KNN-MT模型实现了显着改善，而且表现出更好的鲁棒性。我们的代码可在https://github.com/deeplearnxmu/robust-knn-mt上找到。
k-Nearest-Neighbor Machine Translation (kNN-MT) becomes an important research direction of NMT in recent years. Its main idea is to retrieve useful key-value pairs from an additional datastore to modify translations without updating the NMT model. However, the underlying retrieved noisy pairs will dramatically deteriorate the model performance. In this paper, we conduct a preliminary study and find that this problem results from not fully exploiting the prediction of the NMT model. To alleviate the impact of noise, we propose a confidence-enhanced kNN-MT model with robust training. Concretely, we introduce the NMT confidence to refine the modeling of two important components of kNN-MT: kNN distribution and the interpolation weight. Meanwhile we inject two types of perturbations into the retrieved pairs for robust training. Experimental results on four benchmark datasets demonstrate that our model not only achieves significant improvements over current kNN-MT models, but also exhibits better robustness. Our code is available at https://github.com/DeepLearnXMU/Robust-knn-mt.
