论文标题
插入局部变压器
Insertion-Deletion Transformer
论文作者
论文摘要
我们提出了插入 - 局部变压器，这是一种基于变压器的新型神经结构和序列产生的训练方法。该模型由两个阶段组成，这些阶段是迭代执行的，1）插入阶段和2）缺失阶段。插入阶段参数分布了当前输出假设上的插入分布，而删除阶段则参数分布了当前输出假设上的删除分布。训练方法是一种有原则且简单的算法，其中删除模型从插入模型输出中直接在上方获得其信号。我们证明了插入 - 缺失变压器对合成翻译任务的有效性，从而获得了仅插入模型的显着BLEU得分提高。
We propose the Insertion-Deletion Transformer, a novel transformer-based neural architecture and training method for sequence generation. The model consists of two phases that are executed iteratively, 1) an insertion phase and 2) a deletion phase. The insertion phase parameterizes a distribution of insertions on the current output hypothesis, while the deletion phase parameterizes a distribution of deletions over the current output hypothesis. The training method is a principled and simple algorithm, where the deletion model obtains its signal directly on-policy from the insertion model output. We demonstrate the effectiveness of our Insertion-Deletion Transformer on synthetic translation tasks, obtaining significant BLEU score improvement over an insertion-only model.
