论文标题
使用自由能范式通过子空间进行增强学习
Reinforcement Learning with Subspaces using Free Energy Paradigm
论文作者
论文摘要
在大规模的问题中，标准的增强学习算法遭受了缓慢的学习速度。在本文中，我们遵循使用子空间来解决此问题的框架。我们提出了一个自由化最小化框架，以选择子空间并将状态空间的策略集成到子空间中。我们提出的自由能最小化框架取决于汤普森采样政策和子空间和状态空间的行为政策。因此，它适用于各种任务，离散或连续状态空间，无模型和基于模型的任务。通过一组实验，我们表明该通用框架高度提高了学习速度。我们还提供融合证明。
In large-scale problems, standard reinforcement learning algorithms suffer from slow learning speed. In this paper, we follow the framework of using subspaces to tackle this problem. We propose a free-energy minimization framework for selecting the subspaces and integrate the policy of the state-space into the subspaces. Our proposed free-energy minimization framework rests upon Thompson sampling policy and behavioral policy of subspaces and the state-space. It is therefore applicable to a variety of tasks, discrete or continuous state space, model-free and model-based tasks. Through a set of experiments, we show that this general framework highly improves the learning speed. We also provide a convergence proof.
