论文标题
使用细心网络修剪紧凑的神经表示
Compact Neural Representation Using Attentive Network Pruning
论文作者
论文摘要
深度神经网络已经发展为苛刻的力量，因此很难应用于小型移动平台。网络参数减少方法已被引入系统地处理深网的计算和内存复杂性。我们建议研究专注连接修剪来处理神经网络减少冗余的能力，以促进计算需求的减少。在这项工作中，我们描述了一种自上而下的注意机制，该机制被添加到自下而上的前馈网络中，以选择重要的连接，然后在所有参数层上修剪冗余。我们的方法不仅引入了一种新型的分层选择机制作为修剪的基础，而且在实验评估中仍与先前的基线方法保持竞争力。我们使用流行基准数据集上的不同网络体系结构进行实验以显示高压缩比，而准确性丧失可以实现。
Deep neural networks have evolved to become power demanding and consequently difficult to apply to small-size mobile platforms. Network parameter reduction methods have been introduced to systematically deal with the computational and memory complexity of deep networks. We propose to examine the ability of attentive connection pruning to deal with redundancy reduction in neural networks as a contribution to the reduction of computational demand. In this work, we describe a Top-Down attention mechanism that is added to a Bottom-Up feedforward network to select important connections and subsequently prune redundant ones at all parametric layers. Our method not only introduces a novel hierarchical selection mechanism as the basis of pruning but also remains competitive with previous baseline methods in the experimental evaluation. We conduct experiments using different network architectures on popular benchmark datasets to show high compression ratio is achievable with negligible loss of accuracy.
