论文标题
观看您的上流卷：基于CNN的生成深神经网络未能再现光谱分布
Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions
论文作者
论文摘要
生成卷积深神经网络，例如流行的GAN体系结构依靠基于卷积的上采样方法来生成图像或视频序列（例如图像或视频序列）。在本文中，我们表明，常见的上采样方法，即称为上卷积或转置卷积，导致这种模型无法正确地重现自然训练数据的光谱分布。这种效果与潜在的体系结构无关，我们表明它可以用来轻松检测生成的数据，例如公共基准上的Deepfakes，其精度高达100％。 为了克服当前生成模型的这一缺点，我们建议将新型的光谱正则化项添加到训练优化目标中。我们表明，这种方法不仅允许训练避免高频错误的频谱一致的gan。另外，我们表明频谱的正确近似对生成网络的训练稳定性和输出质量具有积极影响。
Generative convolutional deep neural networks, e.g. popular GAN architectures, are relying on convolution based up-sampling methods to produce non-scalar outputs like images or video sequences. In this paper, we show that common up-sampling methods, i.e. known as up-convolution or transposed convolution, are causing the inability of such models to reproduce spectral distributions of natural training data correctly. This effect is independent of the underlying architecture and we show that it can be used to easily detect generated data like deepfakes with up to 100% accuracy on public benchmarks. To overcome this drawback of current generative models, we propose to add a novel spectral regularization term to the training optimization objective. We show that this approach not only allows to train spectral consistent GANs that are avoiding high frequency errors. Also, we show that a correct approximation of the frequency spectrum has positive effects on the training stability and output quality of generative networks.
