论文标题
GLAMA：一般图像插图的关节空间和频率损失
GLaMa: Joint Spatial and Frequency Loss for General Image Inpainting
论文作者
论文摘要
图像介绍的目的是使用其余部分中的上下文信息恢复划痕和损坏区域。近年来，由于卷积神经网络（CNN）的复兴，图像灌输任务取得了巨大的突破。但是，大多数工作都认为面具类型不足，并且在遇到看不见的口罩时，它们的性能会急剧下降。为了应对这些挑战，我们提出了一种简单而通用的方法，以基于喇嘛图像镶嵌框架（称为Glama）解决此问题。我们提出的Glama可以通过使用更多类型的面具来更好地捕获不同类型的缺少信息。通过在训练阶段加入更多退化的图像，我们可以期望增强模型相对于各种掩模的鲁棒性。为了产生更合理的结果，除了传统的空间重建损失和对抗性损失外，我们还进一步引入了基于频率的损失。特别是，我们在空间和频域中引入了有效的重建损失，以减少棋盘效应和重建图像中的波纹。广泛的实验表明，我们的方法可以提高FFHQ，Imagenet，Place2和Wikiart数据集上每种掩码的原始LAMA方法的性能。在NTIRE 2022 IMAGE INPAIRTING挑战曲目1中，无监督的拟议的Glama在PSNR，LPIPS和SSIM方面排名第一。
The purpose of image inpainting is to recover scratches and damaged areas using context information from remaining parts. In recent years, thanks to the resurgence of convolutional neural networks (CNNs), image inpainting task has made great breakthroughs. However, most of the work consider insufficient types of mask, and their performance will drop dramatically when encountering unseen masks. To combat these challenges, we propose a simple yet general method to solve this problem based on the LaMa image inpainting framework, dubbed GLaMa. Our proposed GLaMa can better capture different types of missing information by using more types of masks. By incorporating more degraded images in the training phase, we can expect to enhance the robustness of the model with respect to various masks. In order to yield more reasonable results, we further introduce a frequency-based loss in addition to the traditional spatial reconstruction loss and adversarial loss. In particular, we introduce an effective reconstruction loss both in the spatial and frequency domain to reduce the chessboard effect and ripples in the reconstructed image. Extensive experiments demonstrate that our method can boost the performance over the original LaMa method for each type of mask on FFHQ, ImageNet, Places2 and WikiArt dataset. The proposed GLaMa was ranked first in terms of PSNR, LPIPS and SSIM in the NTIRE 2022 Image Inpainting Challenge Track 1 Unsupervised.
