论文标题
预先训练的令牌重定放置的检测模型是很少的射击学习者
Pre-trained Token-replaced Detection Model as Few-shot Learner
论文作者
论文摘要
预先训练的蒙版语言模型表现出了很少的学习者的能力。在本文中，作为替代方案，我们提出了一种新颖的方法，可以使用Electra（例如Electra）的预先训练的代币重新训练的检测模型来进行几次学习。在这种方法中，我们将分类或回归任务重新制定为代币重新定位的检测问题。具体来说，我们首先为每个任务定义一个模板和标签描述单词，然后将其放入输入中以形成自然语言提示。然后，我们采用预先训练的令牌重新定位的检测模型来预测哪个标签描述单词在提示符中的所有标签描述中最原始的单词（即最少替换）。在16个数据集上进行的系统评估表明，我们的方法在单句子和两句学习任务中都超过了具有预先训练的蒙版语言模型的少数学习者。
Pre-trained masked language models have demonstrated remarkable ability as few-shot learners. In this paper, as an alternative, we propose a novel approach to few-shot learning with pre-trained token-replaced detection models like ELECTRA. In this approach, we reformulate a classification or a regression task as a token-replaced detection problem. Specifically, we first define a template and label description words for each task and put them into the input to form a natural language prompt. Then, we employ the pre-trained token-replaced detection model to predict which label description word is the most original (i.e., least replaced) among all label description words in the prompt. A systematic evaluation on 16 datasets demonstrates that our approach outperforms few-shot learners with pre-trained masked language models in both one-sentence and two-sentence learning tasks.
