论文标题
DAQ:深层图像超分辨率网络的频道分布感知量化
DAQ: Channel-Wise Distribution-Aware Quantization for Deep Image Super-Resolution Networks
论文作者
论文摘要
量化深层卷积神经网络的图像超分辨率大大降低了其计算成本。但是,现有作品要么在4个或更低的位宽度的超低精度下遭受严重的性能下降,要么需要进行大量的微调过程才能恢复性能。据我们所知,这种对低精确度的脆弱性取决于特征映射值的两个统计观察。首先,特征映射值的分布每个通道和每个输入图像都有显着变化。其次,特征地图的离群值可以主导量化误差。基于这些观察结果,我们提出了一种新颖的分布感知量化方案(DAQ),该方案促进了超低精度准确的无训练量化。 DAQ的一个简单函数确定了具有低计算负担的特征图和权重的动态范围。此外,我们的方法可以通过计算每个通道的相对灵敏度,而无需涉及任何训练过程,从而实现混合精确量化。尽管如此,量化感知培训也适用于辅助性能增益。我们的新方法以超低精度优于最新的图像超分辨率网络,优于最新的无培训甚至基于培训的量化方法。
Quantizing deep convolutional neural networks for image super-resolution substantially reduces their computational costs. However, existing works either suffer from a severe performance drop in ultra-low precision of 4 or lower bit-widths, or require a heavy fine-tuning process to recover the performance. To our knowledge, this vulnerability to low precisions relies on two statistical observations of feature map values. First, distribution of feature map values varies significantly per channel and per input image. Second, feature maps have outliers that can dominate the quantization error. Based on these observations, we propose a novel distribution-aware quantization scheme (DAQ) which facilitates accurate training-free quantization in ultra-low precision. A simple function of DAQ determines dynamic range of feature maps and weights with low computational burden. Furthermore, our method enables mixed-precision quantization by calculating the relative sensitivity of each channel, without any training process involved. Nonetheless, quantization-aware training is also applicable for auxiliary performance gain. Our new method outperforms recent training-free and even training-based quantization methods to the state-of-the-art image super-resolution networks in ultra-low precision.