论文标题
抽象摘要模型中定量值的探测
Probing of Quantitative Values in Abstractive Summarization Models
论文作者
论文摘要
抽象性文本摘要最近已成为一种流行的方法，但是数据幻觉仍然是一个严重的问题，包括使用定量数据。我们提出了一组探测测试，以评估抽象摘要模型对输入文本中发现的定量值的建模的功效。我们的结果表明，在大多数情况下，最近表现的模型的编码器难以提供与基线相比，在输入中充分代表定量值的嵌入，尤其是，它们在某些情况下的表现优于某些情况，但令人惊讶的是，并非全部。在我们的假设下，这表明编码器的性能有助于数量幻觉问题。特别是一种模型类型，即Distilbart-CDM，在多个实验中观察到表现不佳的随机初始化表示形式，并且性能与BERT表明，摘要任务的标准预处理和微调方法可能在某些编码者的表现不佳中起作用。
Abstractive text summarization has recently become a popular approach, but data hallucination remains a serious problem, including with quantitative data. We propose a set of probing tests to evaluate the efficacy of abstract summarization models' modeling of quantitative values found in the input text. Our results show that in most cases, the encoders of recent SOTA-performing models struggle to provide embeddings that adequately represent quantitative values in the input compared to baselines, and in particular, they outperform random representations in some, but surprisingly not all, cases. Under our assumptions, this suggests that the encoder's performance contributes to the quantity hallucination problem. One model type in particular, DistilBART-CDM, was observed to underperform randomly initialized representations for several experiments, and performance versus BERT suggests that standard pretraining and fine-tuning approaches for the summarization task may play a role in underperformance for some encoders.
