论文标题
comfact：链接上下文常识知识的基准
ComFact: A Benchmark for Linking Contextual Commonsense Knowledge
论文作者
论文摘要
了解丰富的叙述，例如对话和故事，通常需要自然语言处理系统来从常识知识图中获取相关知识。但是，这些系统通常使用简单的启发式方法从KGS中获取事实，这些启发式方法无视识别与处境相关的常识性知识（例如，情境化，隐性，模棱两可）的复杂挑战。 在这项工作中，我们提出了通用事实联系的新任务，在该任务中，给出了上下文并接受了培训，以确定与库格的情况相关的常识性知识。我们的小说基准comfact包含〜293k的中文相关性注释，对四个风格上不同的对话和讲故事的数据集进行了共识三胞胎的相关性注释。实验结果证实，启发式事实联系方法是不精确的知识提取器。学到的事实链接模型表明了这些启发式方法的全面性能改进（〜34.6％F1）。此外，改进的知识检索为对话响应生成任务提供了9.8％的平均下游改善。但是，事实联系模型的表现仍然显着不足，这表明我们的基准是NLP系统增强常识性研究的有前途的测试床。
Understanding rich narratives, such as dialogues and stories, often requires natural language processing systems to access relevant knowledge from commonsense knowledge graphs. However, these systems typically retrieve facts from KGs using simple heuristics that disregard the complex challenges of identifying situationally-relevant commonsense knowledge (e.g., contextualization, implicitness, ambiguity). In this work, we propose the new task of commonsense fact linking, where models are given contexts and trained to identify situationally-relevant commonsense knowledge from KGs. Our novel benchmark, ComFact, contains ~293k in-context relevance annotations for commonsense triplets across four stylistically diverse dialogue and storytelling datasets. Experimental results confirm that heuristic fact linking approaches are imprecise knowledge extractors. Learned fact linking models demonstrate across-the-board performance improvements (~34.6% F1) over these heuristics. Furthermore, improved knowledge retrieval yielded average downstream improvements of 9.8% for a dialogue response generation task. However, fact linking models still significantly underperform humans, suggesting our benchmark is a promising testbed for research in commonsense augmentation of NLP systems.
