论文标题
EventHands：实时神经3D手姿势估算事件流
EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream
论文作者
论文摘要
来自单眼视频的3D手姿势估算是一个长期且具有挑战性的问题，现在看到了强劲的上涨。在这项工作中，我们首次使用单个事件摄像机来解决它，即，异步视觉传感器对亮度变化做出反应。我们的EventHands方法具有以前没有使用单个RGB或深度摄像头证明的特征，例如低数据吞吐量的高时间分辨率和1000 Hz时的实时性能。由于与经典摄像机相比，事件摄像机的数据模式不同，因此无法直接应用并重新训练事件流的现有方法。因此，我们设计了一种新的神经方法，该方法接受适合学习的新事件流表示，该表示对新生成的合成事件流进行了培训，并可以推广到真实数据。实验表明，就精度及其捕获前所未有的速度手动运动的能力而言，使用颜色（或深度）摄像头的近期单眼方法的表现优于最新的单眼方法。我们的方法，事件流模拟器和数据集公开可用；请参阅https://4dqv.mpi-inf.mpg.de/eventhands/
3D hand pose estimation from monocular videos is a long-standing and challenging problem, which is now seeing a strong upturn. In this work, we address it for the first time using a single event camera, i.e., an asynchronous vision sensor reacting on brightness changes. Our EventHands approach has characteristics previously not demonstrated with a single RGB or depth camera such as high temporal resolution at low data throughputs and real-time performance at 1000 Hz. Due to the different data modality of event cameras compared to classical cameras, existing methods cannot be directly applied to and re-trained for event streams. We thus design a new neural approach which accepts a new event stream representation suitable for learning, which is trained on newly-generated synthetic event streams and can generalise to real data. Experiments show that EventHands outperforms recent monocular methods using a colour (or depth) camera in terms of accuracy and its ability to capture hand motions of unprecedented speed. Our method, the event stream simulator and the dataset are publicly available; see https://4dqv.mpi-inf.mpg.de/EventHands/
