论文标题
调查单词嵌入的频率失真及其对偏置指标的影响
Investigating the Frequency Distortion of Word Embeddings and Its Impact on Bias Metrics
论文作者
论文摘要
最近的研究表明，静态单词嵌入可以编码单词频率信息。但是，几乎没有研究这种现象及其对下游任务的影响。在目前的工作中，我们系统地研究了几个静态单词嵌入中频率和语义相似性之间的关联。我们发现，与其他频率组合之间相比，Skip-gram，Glove和FastText嵌入往往会在高频单词之间产生更高的语义相似性。我们表明，当单词被随机洗牌时，频率和相似性之间的关联也会出现。这证明发现的模式不是由于文本中存在的真实语义关联所致，而是嵌入一词产生的伪影。最后，我们提供了一个示例，说明了单词频率如何通过基于嵌入的指标强烈影响性别偏见的测量。特别是，我们进行了一个受控的实验，表明偏见甚至可以通过操纵单词频率来改变符号或扭转其顺序。
Recent research has shown that static word embeddings can encode word frequency information. However, little has been studied about this phenomenon and its effects on downstream tasks. In the present work, we systematically study the association between frequency and semantic similarity in several static word embeddings. We find that Skip-gram, GloVe and FastText embeddings tend to produce higher semantic similarity between high-frequency words than between other frequency combinations. We show that the association between frequency and similarity also appears when words are randomly shuffled. This proves that the patterns found are not due to real semantic associations present in the texts, but are an artifact produced by the word embeddings. Finally, we provide an example of how word frequency can strongly impact the measurement of gender bias with embedding-based metrics. In particular, we carry out a controlled experiment that shows that biases can even change sign or reverse their order by manipulating word frequencies.
