论文标题
主动预测编码：学习层次结构世界模型的统一神经框架
Active Predictive Coding: A Unified Neural Framework for Learning Hierarchical World Models for Perception and Planning
论文作者
论文摘要
预测性编码已成为大脑如何通过预测学习的重要模型，并预测了最近AI架构（例如变形金刚）中预测性学习的重要性。在这里，我们为预测编码提供了一个新的框架，称为主动预测编码，它可以学习层次结构世界模型并在AI中解决两个根本不同的开放问题：（1）我们如何学习组成表示，例如零件 - 全部层次结构，for ecorivariant视觉？ （2）我们如何通过从原始政策中撰写复杂的动作序列来解决大规模的计划问题，这对于传统的强化学习很难？我们的方法利用了超网络，自学的学习和强化学习学习层次结构世界模型，这些模型将任务不变的状态过渡网络和任务依赖性策略网络在多个抽象级别上结合在一起。我们在各种视觉数据集（MNIST，FashionMnist，Omniglot）以及可扩展的层次结构计划问题上证明了方法的生存能力。据我们所知，我们的结果代表了Hinton提出的零件学习问题的统一解决方案，Hawkins提出的嵌套参考帧问题以及在加强学习中提出的综合国家行动层次学习问题。
Predictive coding has emerged as a prominent model of how the brain learns through predictions, anticipating the importance accorded to predictive learning in recent AI architectures such as transformers. Here we propose a new framework for predictive coding called active predictive coding which can learn hierarchical world models and solve two radically different open problems in AI: (1) how do we learn compositional representations, e.g., part-whole hierarchies, for equivariant vision? and (2) how do we solve large-scale planning problems, which are hard for traditional reinforcement learning, by composing complex action sequences from primitive policies? Our approach exploits hypernetworks, self-supervised learning and reinforcement learning to learn hierarchical world models that combine task-invariant state transition networks and task-dependent policy networks at multiple abstraction levels. We demonstrate the viability of our approach on a variety of vision datasets (MNIST, FashionMNIST, Omniglot) as well as on a scalable hierarchical planning problem. Our results represent, to our knowledge, the first demonstration of a unified solution to the part-whole learning problem posed by Hinton, the nested reference frames problem posed by Hawkins, and the integrated state-action hierarchy learning problem in reinforcement learning.
