论文标题
机器人心电图：通过在YouTube上观看人类来学习机器人手工模仿者
Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtube
论文作者
论文摘要
我们构建了一个系统，可以通过自己的手展示动作来控制机器人手和手臂。机器人通过单个RGB摄像机观察人类操作员，并实时模仿他们的动作。人的手和机器人的手在形状，大小和关节结构上有所不同，并且从单个未校准的相机中进行这种翻译是一个高度不受约束的问题。此外，重新定位的轨迹必须有效地在物理机器人上执行任务，这要求它们在时间上平稳且没有自我挑战。我们的主要见解是，尽管配对的人类机器人对应数据的收集价格很高，但互联网包含大量丰富而多样的人类手视频的语料库。我们利用这些数据来训练一个理解人手并将人类视频流的系统训练，并将人类视频流重新定为机器人手臂轨迹，该轨迹光滑，迅速，安全和语义与指导演示相似。我们证明，它使以前未经训练的人能够在各种灵活的操纵任务上对机器人进行遗传。我们的低成本，无手套，无标记的远程远程操作系统使机器人教学更容易访问，我们希望它可以帮助机器人学习在现实世界中自主行动。 https://robotic-telekinesis.github.io/的视频
We build a system that enables any human to control a robot hand and arm, simply by demonstrating motions with their own hand. The robot observes the human operator via a single RGB camera and imitates their actions in real-time. Human hands and robot hands differ in shape, size, and joint structure, and performing this translation from a single uncalibrated camera is a highly underconstrained problem. Moreover, the retargeted trajectories must effectively execute tasks on a physical robot, which requires them to be temporally smooth and free of self-collisions. Our key insight is that while paired human-robot correspondence data is expensive to collect, the internet contains a massive corpus of rich and diverse human hand videos. We leverage this data to train a system that understands human hands and retargets a human video stream into a robot hand-arm trajectory that is smooth, swift, safe, and semantically similar to the guiding demonstration. We demonstrate that it enables previously untrained people to teleoperate a robot on various dexterous manipulation tasks. Our low-cost, glove-free, marker-free remote teleoperation system makes robot teaching more accessible and we hope that it can aid robots in learning to act autonomously in the real world. Videos at https://robotic-telekinesis.github.io/
