论文标题
预先训练的语言模型作为汽车投诉分析的知识库
Pre-trained language models as knowledge bases for Automotive Complaint Analysis
论文作者
论文摘要
最近，已经显示出诸如Bert（Devlin等，2018）之类的大型预训练的语言模型能够存储在其前训练语料库中捕获的常识性事实知识（Petroni等，2019）。在我们的工作中，我们进一步评估了该行业的应用程序，创建了一组专门设计的探针，该探针旨在揭示从汽车行业中非结构化客户反馈中描述的事件捕获的技术质量问题。在使用填充掩盖任务的预训练模型的开箱即用版本后，我们通过在缺陷调查办公室（ODI）投诉数据集中动态提供更多知识。在我们的实验中，与Petroni等人一样，与对事实知识本身的查询相比，模型在域特异性主题上的查询表现出了性能。 （2019年）已经完成了。对于大多数评估的架构，预测正确的令牌是用$ precision@1 $（$ p@1 $）高于60 \％的，而对于$ p@5 $和$ p@10 $偶数均高于80 \％\％\％，最多达到90 \％。这些结果表明，使用语言模型作为客户反馈的结构化分析的知识库的潜力。
Recently it has been shown that large pre-trained language models like BERT (Devlin et al., 2018) are able to store commonsense factual knowledge captured in its pre-training corpus (Petroni et al., 2019). In our work we further evaluate this ability with respect to an application from industry creating a set of probes specifically designed to reveal technical quality issues captured as described incidents out of unstructured customer feedback in the automotive industry. After probing the out-of-the-box versions of the pre-trained models with fill-in-the-mask tasks we dynamically provide it with more knowledge via continual pre-training on the Office of Defects Investigation (ODI) Complaints data set. In our experiments the models exhibit performance regarding queries on domain-specific topics compared to when queried on factual knowledge itself, as Petroni et al. (2019) have done. For most of the evaluated architectures the correct token is predicted with a $Precision@1$ ($P@1$) of above 60\%, while for $P@5$ and $P@10$ even values of well above 80\% and up to 90\% respectively are reached. These results show the potential of using language models as a knowledge base for structured analysis of customer feedback.
