论文标题
Pac-Bayes压缩的界限如此之紧，以至于可以解释概括
PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization
论文作者
论文摘要
尽管在为深度神经网络开发非呈现概括范围方面取得了进展，但这些界限往往对深度学习的作用往往是不明智的。在本文中，我们基于量化线性子空间中神经网络参数的压缩方法，深刻改进了先前的结果，以在包括转移学习在内的各种任务上提供最新的概括界限。我们使用这些紧密的界限来更好地理解模型大小，模棱两可的作用，以及优化的隐性偏见，以在深度学习中进行概括。值得注意的是，我们发现大型模型可以比以前已知的更大程度地压缩，封装了Occam的剃须刀。我们还主张在解释概括方面依赖于数据。
While there has been progress in developing non-vacuous generalization bounds for deep neural networks, these bounds tend to be uninformative about why deep learning works. In this paper, we develop a compression approach based on quantizing neural network parameters in a linear subspace, profoundly improving on previous results to provide state-of-the-art generalization bounds on a variety of tasks, including transfer learning. We use these tight bounds to better understand the role of model size, equivariance, and the implicit biases of optimization, for generalization in deep learning. Notably, we find large models can be compressed to a much greater extent than previously known, encapsulating Occam's razor. We also argue for data-independent bounds in explaining generalization.
