论文标题
Word2Vec的基础光谱
The Spectral Underpinning of word2vec
论文作者
论文摘要
由于Mikolov \ textit {et al。}（2013），Word2Vec是一种单词嵌入方法，可在自然语言处理中广泛使用。尽管它取得了巨大的成功和频繁的使用，但仍然缺乏理论上的理由。我们论文的主要贡献是对Word2Vec的高度非线性功能进行严格的分析。我们的结果表明，Word2Vec可能主要是由潜在的光谱方法驱动的。这种见解可能为获得Word2Vec的可证明的保证打开了大门。我们通过数值模拟来支持这些发现。一个令人着迷的开放问题是，光谱方法未捕获的Word2Vec的非线性特性是否有益，如果是的，则通过什么机制是有益的。
word2vec due to Mikolov \textit{et al.} (2013) is a word embedding method that is widely used in natural language processing. Despite its great success and frequent use, theoretical justification is still lacking. The main contribution of our paper is to propose a rigorous analysis of the highly nonlinear functional of word2vec. Our results suggest that word2vec may be primarily driven by an underlying spectral method. This insight may open the door to obtaining provable guarantees for word2vec. We support these findings by numerical simulations. One fascinating open question is whether the nonlinear properties of word2vec that are not captured by the spectral method are beneficial and, if so, by what mechanism.
