论文标题
模拟四足代理的分层分散的深钢筋学习体系结构
Hierarchical Decentralized Deep Reinforcement Learning Architecture for a Simulated Four-Legged Agent
论文作者
论文摘要
腿部运动在本质上是广泛的，并启发了当前机器人的设计。这些腿机器人的控制器通常被实现为一个集中式实例。但是，在本质上，对运动的控制以分层和分散的方式发生。将这些生物设计原理引入机器人控制系统已激发了这项工作。我们解决了一个问题，分散和分层控制是否对腿部机器人有益，并提出了一种新颖的分散，分层的结构，以控制模拟的腿部代理。复杂性不同的三个不同任务旨在基准五个架构（集中式，分散，分层和分层分散式体系结构的两种不同组合）。结果表明，分散层次结构的分散层次有助于对代理的学习，确保更节能的运动以及对新看不见的环境的鲁棒性。此外，此比较阐明了模块化在层次结构中解决复杂目标指导任务的重要性。我们提供架构（https://github.com/wzaielamri/hddrl）的开源代码实现。
Legged locomotion is widespread in nature and has inspired the design of current robots. The controller of these legged robots is often realized as one centralized instance. However, in nature, control of movement happens in a hierarchical and decentralized fashion. Introducing these biological design principles into robotic control systems has motivated this work. We tackle the question whether decentralized and hierarchical control is beneficial for legged robots and present a novel decentral, hierarchical architecture to control a simulated legged agent. Three different tasks varying in complexity are designed to benchmark five architectures (centralized, decentralized, hierarchical and two different combinations of hierarchical decentralized architectures). The results demonstrate that decentralizing the different levels of the hierarchical architectures facilitates learning of the agent, ensures more energy efficient movements as well as robustness towards new unseen environments. Furthermore, this comparison sheds light on the importance of modularity in hierarchical architectures to solve complex goal-directed tasks. We provide an open-source code implementation of our architecture (https://github.com/wzaielamri/hddrl).
