论文标题
建立生成对抗网络的多元化和忠实的一声改编
Towards Diverse and Faithful One-shot Adaption of Generative Adversarial Networks
论文作者
论文摘要
一击生成域Adaption旨在仅使用一个参考图像将一个预训练的发电机传递到一个新域中。但是,适用的生成器(i)生成从预训练的生成器继承的多种图像而(ii)(ii)忠实地获取参考图像的特定域特定属性和样式的不同图像仍然非常具有挑战性。在本文中,我们提出了一种新颖的单发性生成域自适应方法,即Difa,以进行多样化和忠实的适应。对于全局级适应,我们利用参考图像的剪辑嵌入与源图像的平均嵌入之间的差异来限制目标发生器。对于本地级别的适应,我们引入了一个细心的样式损失,该损失将每个适应图像的中间令牌与参考图像的相应令牌保持一致。为了促进多元化的生成,引入了选择性跨域一致性,以选择和保留域共享属性,以编辑潜在的$ \ Mathcal {W}+$ $空间来继承预训练的生成器的多样性。广泛的实验表明,我们的方法在定量和定性上都优于最先进的实验,尤其是对于大域间隙的情况。此外,我们的DIFA可以轻松地扩展到零击生成域的适应性,并具有吸引人的结果。代码可从https://github.com/1170300521/difa获得。
One-shot generative domain adaption aims to transfer a pre-trained generator on one domain to a new domain using one reference image only. However, it remains very challenging for the adapted generator (i) to generate diverse images inherited from the pre-trained generator while (ii) faithfully acquiring the domain-specific attributes and styles of the reference image. In this paper, we present a novel one-shot generative domain adaption method, i.e., DiFa, for diverse generation and faithful adaptation. For global-level adaptation, we leverage the difference between the CLIP embedding of reference image and the mean embedding of source images to constrain the target generator. For local-level adaptation, we introduce an attentive style loss which aligns each intermediate token of adapted image with its corresponding token of the reference image. To facilitate diverse generation, selective cross-domain consistency is introduced to select and retain the domain-sharing attributes in the editing latent $\mathcal{W}+$ space to inherit the diversity of pre-trained generator. Extensive experiments show that our method outperforms the state-of-the-arts both quantitatively and qualitatively, especially for the cases of large domain gaps. Moreover, our DiFa can easily be extended to zero-shot generative domain adaption with appealing results. Code is available at https://github.com/1170300521/DiFa.