论文标题
针对自然和现实的对抗表扰动，文本到SQL模型的稳健性
Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation
论文作者
论文摘要
文本到SQL解析器对对抗扰动的鲁棒性在提供高度可靠的应用中起着至关重要的作用。先前沿着这条线的研究主要集中在自然语言问题方面的扰动，忽略了表的可变性。在此激励的情况下，我们提出了对抗表扰动（ATP）作为一种新的攻击范式，以衡量文本到SQL模型的鲁棒性。按照这个主张，我们策划了Adveta，这是具有自然和现实ATP的首个鲁棒性评估基准。所有经过测试过的最先进的模型都会在Adveta上出现巨大的性能下降，从而揭示了模型在现实世界实践中的脆弱性。为了防御ATP，我们建立了一个系统的对抗训练示例生成框架，该框架量身定制，以更好地对表格数据进行上下文化。实验表明，我们的方法不仅为桌面扰动带来了最佳的鲁棒性改进，而且还可以实质性地反对NL侧扰动。我们在：https：//github.com/microsoft/contextualsp上发布基准和代码。
The robustness of Text-to-SQL parsers against adversarial perturbations plays a crucial role in delivering highly reliable applications. Previous studies along this line primarily focused on perturbations in the natural language question side, neglecting the variability of tables. Motivated by this, we propose the Adversarial Table Perturbation (ATP) as a new attacking paradigm to measure the robustness of Text-to-SQL models. Following this proposition, we curate ADVETA, the first robustness evaluation benchmark featuring natural and realistic ATPs. All tested state-of-the-art models experience dramatic performance drops on ADVETA, revealing models' vulnerability in real-world practices. To defend against ATP, we build a systematic adversarial training example generation framework tailored for better contextualization of tabular data. Experiments show that our approach not only brings the best robustness improvement against table-side perturbations but also substantially empowers models against NL-side perturbations. We release our benchmark and code at: https://github.com/microsoft/ContextualSP.
