论文标题
发育序列的奇怪案例：关于稀疏，转移学习，概括和大脑
The curious case of developmental BERTology: On sparsity, transfer learning, generalization and the brain
论文作者
论文摘要
在本文中，我们通过大型语言模型，转移学习和网络压缩探索深度学习与神经科学之间的相交点。就像知觉和认知神经生理学一样，它启发了有效的深层神经网络体系结构，这反过来又成为理解大脑的有用模型，在这里，我们探讨了生物学神经发育如何激发有效且可靠的优化程序，从而可以作为大脑成熟和衰老的有用模型。
In this essay, we explore a point of intersection between deep learning and neuroscience, through the lens of large language models, transfer learning and network compression. Just like perceptual and cognitive neurophysiology has inspired effective deep neural network architectures which in turn make a useful model for understanding the brain, here we explore how biological neural development might inspire efficient and robust optimization procedures which in turn serve as a useful model for the maturation and aging of the brain.
