论文标题
语言知识融合在语言理解任务中的经验重新审视
An Empirical Revisiting of Linguistic Knowledge Fusion in Language Understanding Tasks
论文作者
论文摘要
尽管在大规模的语言模型预处理中出现了语言知识，但最近的工作试图将人类定义的语言先验纳入特定于任务的微调中。从解析器中注入语言模型或语义知识已显示出许多语言理解任务的改进。为了进一步研究结构性语言先验的有效性，我们对用微不足道的图形或树木（很少携带语言知识（例如平衡的树）来代替解析图或树木的经验研究）用于胶水基准中的任务。用微不足道的图编码可以在完全监督和少数拍摄的设置中实现竞争性甚至更好的性能。它表明，收益可能并非显着归因于明确的语言先验，而是融合层带来的更多特征相互作用。因此，我们呼吁人们注意将来将琐碎的图作为必要的基线来设计高级知识融合方法。
Though linguistic knowledge emerges during large-scale language model pretraining, recent work attempt to explicitly incorporate human-defined linguistic priors into task-specific fine-tuning. Infusing language models with syntactic or semantic knowledge from parsers has shown improvements on many language understanding tasks. To further investigate the effectiveness of structural linguistic priors, we conduct empirical study of replacing parsed graphs or trees with trivial ones (rarely carrying linguistic knowledge e.g., balanced tree) for tasks in the GLUE benchmark. Encoding with trivial graphs achieves competitive or even better performance in fully-supervised and few-shot settings. It reveals that the gains might not be significantly attributed to explicit linguistic priors but rather to more feature interactions brought by fusion layers. Hence we call for attention to using trivial graphs as necessary baselines to design advanced knowledge fusion methods in the future.
