论文标题
Pranc：用于压实深层模型的伪随机网络
PRANC: Pseudo RAndom Networks for Compacting deep models
论文作者
论文摘要
我们证明，深层模型可以用作重量空间中几种随机初始化和冷冻深模型的线性组合。在培训期间，我们寻求位于这些随机模型（即“基础”网络）跨越子空间内的本地最小值。我们的框架，Pranc，可实现深层模型的重大压实。该模型可以使用单个标量“种子”重建，用于生成伪随机网络，以及学习的线性混合系数。 在实际应用中，Pranc解决了有效存储和交流深层模型的挑战，这是在多种情况下的常见瓶颈，包括多门学会学习，持续的学习者，联合系统和边缘设备等。在这项研究中，我们采用Pranc来凝结图像分类模型，并通过压实其相关的隐式神经网络来压缩图像。 Pranc在压缩深层型号$ 100 $ $时，图像分类的优于基准。此外，我们表明Pranc通过即时生成层的权重来实现记忆有效的推理。 pranc的源代码在这里：\ url {https://github.com/ucdvision/pranc}
We demonstrate that a deep model can be reparametrized as a linear combination of several randomly initialized and frozen deep models in the weight space. During training, we seek local minima that reside within the subspace spanned by these random models (i.e., `basis' networks). Our framework, PRANC, enables significant compaction of a deep model. The model can be reconstructed using a single scalar `seed,' employed to generate the pseudo-random `basis' networks, together with the learned linear mixture coefficients. In practical applications, PRANC addresses the challenge of efficiently storing and communicating deep models, a common bottleneck in several scenarios, including multi-agent learning, continual learners, federated systems, and edge devices, among others. In this study, we employ PRANC to condense image classification models and compress images by compacting their associated implicit neural networks. PRANC outperforms baselines with a large margin on image classification when compressing a deep model almost $100$ times. Moreover, we show that PRANC enables memory-efficient inference by generating layer-wise weights on the fly. The source code of PRANC is here: \url{https://github.com/UCDvision/PRANC}
