论文标题
在预训练的语言模型中分析单个神经元
Analyzing Individual Neurons in Pre-trained Language Models
论文作者
论文摘要
虽然已经进行了大量分析来证明在深度NLP模型中所学的表述所捕获的语言知识，但对单个神经元的表达几乎没有关注。我们使用预测形态学，语法和语义的核心语言级分析进行淘汰神经元级别的分析，在诸如：i）的神经元中捕获诸如pre nocation for-n for-li for-li for-n li for-li f lie for l li f lie for-li for l li for l li li f in for：i） ii）网络的哪一部分更多地了解某些语言现象？ iii）信息是如何分布或集中的？ iv）各种架构在学习这些属性方面有何不同？我们发现，与预测语法的较高级别的任务相比，我们发现了较低的神经元任务（例如形态学）的较小的神经元子集，以预测语言任务。我们的研究还揭示了有趣的跨建筑比较。例如，我们发现XLNET中的神经元与BERT和其他人相比，在预测属性时，神经元更加本地化和不相交。
While a lot of analysis has been carried to demonstrate linguistic knowledge captured by the representations learned within deep NLP models, very little attention has been paid towards individual neurons.We carry outa neuron-level analysis using core linguistic tasks of predicting morphology, syntax and semantics, on pre-trained language models, with questions like: i) do individual neurons in pre-trained models capture linguistic information? ii) which parts of the network learn more about certain linguistic phenomena? iii) how distributed or focused is the information? and iv) how do various architectures differ in learning these properties? We found small subsets of neurons to predict linguistic tasks, with lower level tasks (such as morphology) localized in fewer neurons, compared to higher level task of predicting syntax. Our study also reveals interesting cross architectural comparisons. For example, we found neurons in XLNet to be more localized and disjoint when predicting properties compared to BERT and others, where they are more distributed and coupled.
