论文标题
提示：通过提示改进Bert句子嵌入
PromptBERT: Improving BERT Sentence Embeddings with Prompts
论文作者
论文摘要
我们提出了一种提示，这是一种新颖的对比学习方法，用于学习更好的句子表示。我们首先分析了原始BERT嵌入当前句子的缺点，并发现它主要是由于静态令牌嵌入偏见和无效的BERT层。然后，我们提出了第一个基于快速的句子嵌入方法，并讨论了两个表示方法的提示和三个提示方法，以使Bert实现更好的句子嵌入。此外，我们通过模板DeNoising技术提出了一个新颖的无监督培训目标，该目标大大缩短了受监督和无监督环境之间的性能差距。广泛的实验显示了我们方法的有效性。与SIMCSE相比，提示在无监督环境中，基于Bert和Roberta的提示将获得2.29和2.58的改进点。
We propose PromptBERT, a novel contrastive learning method for learning better sentence representation. We firstly analyze the drawback of current sentence embedding from original BERT and find that it is mainly due to the static token embedding bias and ineffective BERT layers. Then we propose the first prompt-based sentence embeddings method and discuss two prompt representing methods and three prompt searching methods to make BERT achieve better sentence embeddings. Moreover, we propose a novel unsupervised training objective by the technology of template denoising, which substantially shortens the performance gap between the supervised and unsupervised settings. Extensive experiments show the effectiveness of our method. Compared to SimCSE, PromptBert achieves 2.29 and 2.58 points of improvement based on BERT and RoBERTa in the unsupervised setting.
