论文标题

使用几个基于贴片的培训的交互式视频风格化

Interactive Video Stylization Using Few-Shot Patch-Based Training

论文作者

Texler, Ondřej, Futschik, David, Kučera, Michal, Jamriška, Ondřej, Sochorová, Šárka, Chai, Menglei, Tulyakov, Sergey, Sýkora, Daniel

论文摘要

在本文中,我们向基于密钥帧的视频风格提出了一种基于学习的方法,该方法使艺术家可以从几个选定的关键帧转换为序列的其余部分。它的关键优势是,所产生的风格在语义上是有意义的,即,根据艺术家的意图对移动对象的特定部分进行了风格。与以前的样式转移技术相反,我们的方法不需要任何冗长的预训练过程或大型培训数据集。我们演示了如何仅使用几个风格化的示例从头开始训练外观翻译网络,同时隐含地保留时间一致性。这导致了一个视频风格化框架,该框架支持实时推理,并行处理以及对任意输出框架的随机访问。它还可以从多个密钥帧中合并内容,而无需执行明确的混合操作。我们在各种交互式场景中演示了其实用的实用程序,在该场景中,用户在选定的键盘上进行绘画,并看到她的样式转移到了现有的录制序列或实时视频流中。

In this paper, we present a learning-based method to the keyframe-based video stylization that allows an artist to propagate the style from a few selected keyframes to the rest of the sequence. Its key advantage is that the resulting stylization is semantically meaningful, i.e., specific parts of moving objects are stylized according to the artist's intention. In contrast to previous style transfer techniques, our approach does not require any lengthy pre-training process nor a large training dataset. We demonstrate how to train an appearance translation network from scratch using only a few stylized exemplars while implicitly preserving temporal consistency. This leads to a video stylization framework that supports real-time inference, parallel processing, and random access to an arbitrary output frame. It can also merge the content from multiple keyframes without the need to perform an explicit blending operation. We demonstrate its practical utility in various interactive scenarios, where the user paints over a selected keyframe and sees her style transferred to an existing recorded sequence or a live video stream.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源