论文标题
通过利用3D合成数据的肖像眼镜和阴影去除
Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Data
论文作者
论文摘要
在肖像中，眼镜可能会阻塞面部区域并在脸上产生铸造阴影，从而降低了许多技术的性能，例如面部验证和表达识别。肖像眼镜的去除对于处理这些问题至关重要。但是，完全去除眼镜是具有挑战性的，因为它们引起的照明效果（例如，铸造阴影）通常很复杂。在本文中，我们提出了一个新颖的框架，以去除眼镜及其脸部图像中的铸造阴影。该方法以检测方式起作用，然后检测到眼镜和铸造阴影，然后从图像中删除。由于缺乏用于监督培训的配对数据，我们提出了一个新的合成肖像数据集，并在检测任务和删除任务中都具有中间和最终的监督。此外，我们应用跨域技术来填补合成数据和真实数据之间的空白。据我们所知，提出的技术是第一个同时删除眼镜及其铸造阴影的技术。代码和合成数据集可在https://github.com/storymy/take-frow-eyeglasses上找到。
In portraits, eyeglasses may occlude facial regions and generate cast shadows on faces, which degrades the performance of many techniques like face verification and expression recognition. Portrait eyeglasses removal is critical in handling these problems. However, completely removing the eyeglasses is challenging because the lighting effects (e.g., cast shadows) caused by them are often complex. In this paper, we propose a novel framework to remove eyeglasses as well as their cast shadows from face images. The method works in a detect-then-remove manner, in which eyeglasses and cast shadows are both detected and then removed from images. Due to the lack of paired data for supervised training, we present a new synthetic portrait dataset with both intermediate and final supervisions for both the detection and removal tasks. Furthermore, we apply a cross-domain technique to fill the gap between the synthetic and real data. To the best of our knowledge, the proposed technique is the first to remove eyeglasses and their cast shadows simultaneously. The code and synthetic dataset are available at https://github.com/StoryMY/take-off-eyeglasses.
