论文标题
通过分配上下文嵌入来参考文本主题，朝着可解释的摘要评估
Towards Interpretable Summary Evaluation via Allocation of Contextual Embeddings to Reference Text Topics
论文作者
论文摘要
尽管摘要生成模型最近取得了广泛的进步，但对自动生成的摘要的评估仍然广泛依赖于单分数系统不足以用于透明评估和深入的定性分析。为了弥合这一差距，我们提出了多方面的可解释的摘要评估方法（MISEM），该方法基于摘要的上下文令牌嵌入到参考文本中确定的语义主题的分配。我们进一步为自动化的摘要评估和交互式视觉分析提供了一个解释性工具箱，对摘要评分，主题识别和令牌主题分配。 Misem实现了有希望的.404 Pearson与人类对TAC'08数据集的判断的相关性。
Despite extensive recent advances in summary generation models, evaluation of auto-generated summaries still widely relies on single-score systems insufficient for transparent assessment and in-depth qualitative analysis. Towards bridging this gap, we propose the multifaceted interpretable summary evaluation method (MISEM), which is based on allocation of a summary's contextual token embeddings to semantic topics identified in the reference text. We further contribute an interpretability toolbox for automated summary evaluation and interactive visual analysis of summary scoring, topic identification, and token-topic allocation. MISEM achieves a promising .404 Pearson correlation with human judgment on the TAC'08 dataset.
