论文标题
超越Tabula-rasa：一种用于物理嵌入3D索科班的模块化加固学习方法
Beyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D Sokoban
论文作者
论文摘要
智能机器人需要使用混凝土，时空复杂的感觉信息和电动机控制实现抽象目标。 Tabula Rasa Deep Greenforce Learning（RL）已经根据视觉，抽象或物理推理解决了苛刻的任务，但是解决这些共同的挑战仍然是一个巨大的挑战。穆霍班（Mujoban）是最近未解决这些挑战的一项尚未解决的基准任务，该机器人需要安排由2D Sokoban难题产生的3D仓库。我们探讨了是否可以通过在有意义的计划层次结构中将RL模块组合在一起来解决诸如Mujoban之类的综合任务，在该层次结构中，模块具有与经典机器人体系结构相似的定义明确的角色。与通常基于模型的经典体系结构不同，我们仅使用经过RL或监督学习训练的无模型模块。我们发现，我们的模块化RL方法极大地超过了Mujoban上最新的单片RL代理。此外，当使用其他机器人平台求解相同任务时，可以重复使用学习的模块。我们的结果共同证明了研究对模块化RL设计的重要性。项目网站：https：//sites.google.com/view/modular-rl/
Intelligent robots need to achieve abstract objectives using concrete, spatiotemporally complex sensory information and motor control. Tabula rasa deep reinforcement learning (RL) has tackled demanding tasks in terms of either visual, abstract, or physical reasoning, but solving these jointly remains a formidable challenge. One recent, unsolved benchmark task that integrates these challenges is Mujoban, where a robot needs to arrange 3D warehouses generated from 2D Sokoban puzzles. We explore whether integrated tasks like Mujoban can be solved by composing RL modules together in a sense-plan-act hierarchy, where modules have well-defined roles similarly to classic robot architectures. Unlike classic architectures that are typically model-based, we use only model-free modules trained with RL or supervised learning. We find that our modular RL approach dramatically outperforms the state-of-the-art monolithic RL agent on Mujoban. Further, learned modules can be reused when, e.g., using a different robot platform to solve the same task. Together our results give strong evidence for the importance of research into modular RL designs. Project website: https://sites.google.com/view/modular-rl/
