论文标题
FACTORMATTE：重新定义重组任务的视频垫子
FactorMatte: Redefining Video Matting for Re-Composition Tasks
论文作者
论文摘要
我们提出了“因子垫子”，这是视频综合的替代表述，它更适合重组任务。因子垫子的目的是将视频内容分离为独立组件，每种都可以看到场景的反事实版本，在该版本中，已删除了其他组件的内容。我们表明，因子贴图映射非常适合更一般的贝叶斯框架，该贝叶斯构架是层次之间的复杂条件相互作用。基于此观察结果，我们提出了一种解决因子矩阵问题的方法，该方法即使对于具有复杂的跨层相互作用（例如飞溅，阴影和反射）的视频，也会产生有用的分解。我们的方法是经过训练的每件视频，不需要对外部大数据集进行预训练，也不需要对场景3D结构的了解。我们进行了广泛的实验，并表明我们的方法不仅可以通过复杂的交互作用来解散场景，而且在现有任务（例如经典的视频效果和背景减法）上的表现优于最高方法。此外，我们在一系列下游任务中演示了方法的好处。请参阅我们的项目网页以获取更多详细信息：https：//factormatte.github.io
We propose "factor matting", an alternative formulation of the video matting problem in terms of counterfactual video synthesis that is better suited for re-composition tasks. The goal of factor matting is to separate the contents of video into independent components, each visualizing a counterfactual version of the scene where contents of other components have been removed. We show that factor matting maps well to a more general Bayesian framing of the matting problem that accounts for complex conditional interactions between layers. Based on this observation, we present a method for solving the factor matting problem that produces useful decompositions even for video with complex cross-layer interactions like splashes, shadows, and reflections. Our method is trained per-video and requires neither pre-training on external large datasets, nor knowledge about the 3D structure of the scene. We conduct extensive experiments, and show that our method not only can disentangle scenes with complex interactions, but also outperforms top methods on existing tasks such as classical video matting and background subtraction. In addition, we demonstrate the benefits of our approach on a range of downstream tasks. Please refer to our project webpage for more details: https://factormatte.github.io
