论文标题
快速准确的神经CRF选区解析
Fast and Accurate Neural CRF Constituency Parsing
论文作者
论文摘要
估计概率分布是NLP字段中的核心问题之一。但是，在深度学习（DL）和前二词前时代，与线性链CRF在序列标记任务中的广泛应用不同，很少有作品将树结构CRF应用于选区解析，这主要是由于Inside Inside-Outside Autside算法的复杂性和效率低。这项工作提出了一个快速准确的神经CRF选区解析器。关键思想是通过在GPU上进行直接大张量操作来批准内部算法以进行损失计算，同时避免通过有效的后传播进行梯度计算的外部算法。我们还提出了一种简单的两阶段括号 - 然后标记的解析方法，以进一步提高效率。为了改善解析性能，受依赖性解析的最新进展的启发，我们引入了基于边界表示和Biaffine注意的新评分体系结构，以及有益的辍学策略。 PTB，CTB5.1和CTB7上的实验表明，我们的两阶段CRF解析器在W/ O和W/ BERT的两个设置上都实现了新的最新性能，并且每秒可以分析超过1,000个句子。我们在https://github.com/yzhangcs/crfpar上发布代码。
Estimating probability distribution is one of the core issues in the NLP field. However, in both deep learning (DL) and pre-DL eras, unlike the vast applications of linear-chain CRF in sequence labeling tasks, very few works have applied tree-structure CRF to constituency parsing, mainly due to the complexity and inefficiency of the inside-outside algorithm. This work presents a fast and accurate neural CRF constituency parser. The key idea is to batchify the inside algorithm for loss computation by direct large tensor operations on GPU, and meanwhile avoid the outside algorithm for gradient computation via efficient back-propagation. We also propose a simple two-stage bracketing-then-labeling parsing approach to improve efficiency further. To improve the parsing performance, inspired by recent progress in dependency parsing, we introduce a new scoring architecture based on boundary representation and biaffine attention, and a beneficial dropout strategy. Experiments on PTB, CTB5.1, and CTB7 show that our two-stage CRF parser achieves new state-of-the-art performance on both settings of w/o and w/ BERT, and can parse over 1,000 sentences per second. We release our code at https://github.com/yzhangcs/crfpar.
