论文标题
波尔卡：波兰问题回答数据集
PolQA: Polish Question Answering Dataset
论文作者
论文摘要
最近提出的用于开放域问答的系统（OPENQA）需要大量的培训数据以实现最先进的性能。但是，已知数据注释很耗时，因此获取昂贵。结果，适当的数据集仅适用于少数语言（主要是英语和中文）。在这项工作中，我们介绍并公开发布POLQA，这是第一个用于OpenQA的波兰数据集。它由7,000个问题，87,525个手动标记的证据段落和7,097,322多个候选人段落组成。每个问题都根据其表述，类型以及答案的实体类型进行分类。该资源使我们能够评估不同注释选择对质量检查系统性能的影响，并提出了一种有效的注释策略，该策略将段落检索@10提高到10.55 p.p.同时将注释成本降低了82％。
Recently proposed systems for open-domain question answering (OpenQA) require large amounts of training data to achieve state-of-the-art performance. However, data annotation is known to be time-consuming and therefore expensive to acquire. As a result, the appropriate datasets are available only for a handful of languages (mainly English and Chinese). In this work, we introduce and publicly release PolQA, the first Polish dataset for OpenQA. It consists of 7,000 questions, 87,525 manually labeled evidence passages, and a corpus of over 7,097,322 candidate passages. Each question is classified according to its formulation, type, as well as entity type of the answer. This resource allows us to evaluate the impact of different annotation choices on the performance of the QA system and propose an efficient annotation strategy that increases the passage retrieval accuracy@10 by 10.55 p.p. while reducing the annotation cost by 82%.
