论文标题
用于神经网络修剪的自动图编码器编码器
Auto Graph Encoder-Decoder for Neural Network Pruning
论文作者
论文摘要
模型压缩旨在在具有有限的计算和存储资源的移动设备上部署深层神经网络（DNN）。但是，大多数现有的模型压缩方法都取决于需要域专业知识的手动定义规则。 DNN本质上是计算图，其中包含丰富的结构信息。在本文中，我们旨在从DNNS的结构信息中找到合适的压缩政策。我们提出了一种与图神经网络（GNN）和增强学习（RL）结合使用的自动图编码器模型压缩（AGMC）方法。我们将目标DNN建模为图形，并使用GNN自动学习DNN的嵌入。我们将我们的方法与基于规则的DNN嵌入模型压缩方法进行了比较，以显示我们方法的有效性。结果表明，我们基于学习的DNN嵌入可实现更好的性能和更高的压缩比，并且搜索步骤较少。我们评估了过度参数化和移动友好的DNN的方法，并将我们的方法与手工制作和基于学习的模型压缩方法进行了比较。在诸如Resnet-56之类的参数化DNN上，我们的方法分别优于手工制作和基于学习的方法，分别为$ 4.36 \％$和$ 2.56 \％$ $更高的精度。此外，在Mobilenet-V2上，我们的压缩比比仅$ 0.93 \％$精度损失的最先进方法更高。
Model compression aims to deploy deep neural networks (DNN) on mobile devices with limited computing and storage resources. However, most of the existing model compression methods rely on manually defined rules, which require domain expertise. DNNs are essentially computational graphs, which contain rich structural information. In this paper, we aim to find a suitable compression policy from DNNs' structural information. We propose an automatic graph encoder-decoder model compression (AGMC) method combined with graph neural networks (GNN) and reinforcement learning (RL). We model the target DNN as a graph and use GNN to learn the DNN's embeddings automatically. We compared our method with rule-based DNN embedding model compression methods to show the effectiveness of our method. Results show that our learning-based DNN embedding achieves better performance and a higher compression ratio with fewer search steps. We evaluated our method on over-parameterized and mobile-friendly DNNs and compared our method with handcrafted and learning-based model compression approaches. On over parameterized DNNs, such as ResNet-56, our method outperformed handcrafted and learning-based methods with $4.36\%$ and $2.56\%$ higher accuracy, respectively. Furthermore, on MobileNet-v2, we achieved a higher compression ratio than state-of-the-art methods with just $0.93\%$ accuracy loss.
