论文标题
通过组件对齐来测量和改进文本到SQL的组成概括
Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment
论文作者
论文摘要
在文本到SQL任务中 - 与NLP的大部分内容一样 - 组成概括是一个主要挑战：神经网络与培训和测试分布不同的组成概括斗争。但是，最新的改进尝试是基于单词级合成数据或特定数据集拆分以产生组成偏差的。在这项工作中，我们提出了一个条款级的组成示例生成方法。我们首先将蜘蛛文本到SQL数据集中的句子划分为子句子，并用其相应的SQL子句注释每个子句子，从而产生新的数据集蜘蛛网。然后，我们通过在不同组合中组成蜘蛛 - SS子句子来构建另一个数据集，即蜘蛛网，以测试模型在构图上概括的能力。实验表明，即使在训练过程中可以看到每个子句子，现有模型在评估蜘蛛网时会出现显着的性能降解。为了解决这个问题，我们修改了许多最先进的模型，以训练蜘蛛网的分段数据，我们表明此方法改善了概括性能。
In text-to-SQL tasks -- as in much of NLP -- compositional generalization is a major challenge: neural networks struggle with compositional generalization where training and test distributions differ. However, most recent attempts to improve this are based on word-level synthetic data or specific dataset splits to generate compositional biases. In this work, we propose a clause-level compositional example generation method. We first split the sentences in the Spider text-to-SQL dataset into sub-sentences, annotating each sub-sentence with its corresponding SQL clause, resulting in a new dataset Spider-SS. We then construct a further dataset, Spider-CG, by composing Spider-SS sub-sentences in different combinations, to test the ability of models to generalize compositionally. Experiments show that existing models suffer significant performance degradation when evaluated on Spider-CG, even though every sub-sentence is seen during training. To deal with this problem, we modify a number of state-of-the-art models to train on the segmented data of Spider-SS, and we show that this method improves the generalization performance.
