论文标题
新兴世界表示：探索经过合成任务的序列模型
Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
论文作者
论文摘要
语言模型显示出令人惊讶的功能范围，但是它们的明显能力的来源尚不清楚。这些网络是否仅记住表面统计的集合，还是依赖于生成所看到序列的过程的内部表示？我们通过将GPT模型的变体应用于简单棋盘游戏Othello中的法律移动的任务来调查这个问题。尽管网络对游戏或其规则没有先验知识，但我们发现了董事会状态出现的非线性内部表示的证据。介入实验表明，该表示形式可用于控制网络的输出并创建“潜在显着图”，以帮助用人类术语来解释预测。
Language models show a surprising range of capabilities, but the source of their apparent competence is unclear. Do these networks just memorize a collection of surface statistics, or do they rely on internal representations of the process that generates the sequences they see? We investigate this question by applying a variant of the GPT model to the task of predicting legal moves in a simple board game, Othello. Although the network has no a priori knowledge of the game or its rules, we uncover evidence of an emergent nonlinear internal representation of the board state. Interventional experiments indicate this representation can be used to control the output of the network and create "latent saliency maps" that can help explain predictions in human terms.
