论文标题
无监督视频分解的可变形精神
Deformable Sprites for Unsupervised Video Decomposition
论文作者
论文摘要
我们描述了一种从输入视频中提取动态场景持续元素的方法。我们将每个场景元素表示为\ emph {可变形的sprite}，由三个组成部分组成：1）整个视频的2D纹理图像，2）元素的人均掩码，以及3）非刚性变形，可将纹理图像映射到每个视频帧中。由此产生的分解允许诸如一致的视频编辑之类的应用。可变形的精灵是一种在单个视频上进行优化的视频自动编码模型，并且不需要在大型数据集上进行培训，也不依赖于预训练的模型。此外，我们的方法不需要对象掩码或其他用户输入，并且发现比以前的工作更广泛的对象。我们在标准视频数据集上评估了我们的方法，并在各种互联网视频中展示了定性结果。代码和视频结果可以在https://deformable-sprites.github.io上找到。
We describe a method to extract persistent elements of a dynamic scene from an input video. We represent each scene element as a \emph{Deformable Sprite} consisting of three components: 1) a 2D texture image for the entire video, 2) per-frame masks for the element, and 3) non-rigid deformations that map the texture image into each video frame. The resulting decomposition allows for applications such as consistent video editing. Deformable Sprites are a type of video auto-encoder model that is optimized on individual videos, and does not require training on a large dataset, nor does it rely on pre-trained models. Moreover, our method does not require object masks or other user input, and discovers moving objects of a wider variety than previous work. We evaluate our approach on standard video datasets and show qualitative results on a diverse array of Internet videos. Code and video results can be found at https://deformable-sprites.github.io
