论文标题
在线适应野外一致的网格重建
Online Adaptation for Consistent Mesh Reconstruction in the Wild
论文作者
论文摘要
本文提出了一种算法，以从野外视频中重建暂时一致的3D网眼。不需要为每个视频框架提供3D网格，2D关键点或相机姿势的注释，我们将基于视频的重建作为一个自我监督的在线适应问题，该问题应用于任何传入的测试视频。我们首先从同一类别的单视图像集合中学习一个特定于类别的3D重建模型，该模型共同预测了图像的形状，纹理和摄像头。然后，在推理时，我们使用自我监督的正则化术语将模型调整为测试视频，从而利用对象实例的时间一致性来强制执行所有重建的网格共享一个共同的纹理图，基本形状以及零件。我们证明，我们的算法从非刚性物体的视频中恢复了时间一致且可靠的3D结构，包括在野外捕获的动物的视频，这是一项极具挑战性的任务。
This paper presents an algorithm to reconstruct temporally consistent 3D meshes of deformable object instances from videos in the wild. Without requiring annotations of 3D mesh, 2D keypoints, or camera pose for each video frame, we pose video-based reconstruction as a self-supervised online adaptation problem applied to any incoming test video. We first learn a category-specific 3D reconstruction model from a collection of single-view images of the same category that jointly predicts the shape, texture, and camera pose of an image. Then, at inference time, we adapt the model to a test video over time using self-supervised regularization terms that exploit temporal consistency of an object instance to enforce that all reconstructed meshes share a common texture map, a base shape, as well as parts. We demonstrate that our algorithm recovers temporally consistent and reliable 3D structures from videos of non-rigid objects including those of animals captured in the wild -- an extremely challenging task rarely addressed before.
