论文标题
适应现代深度学习的线性拉普拉斯模型证据
Adapting the Linearised Laplace Model Evidence for Modern Deep Learning
论文作者
论文摘要
估计模型不确定性的线性拉普拉斯方法在贝叶斯深度学习社区中引起了人们的重新关注。该方法提供了可靠的误差线，并接受模型证据的封闭式表达式，从而允许对模型超参数的可扩展选择。在这项工作中，我们检查了这种方法背后的假设，尤其是与模型选择结合在一起。我们表明，这些与一些深度学习的标准工具（具有深度学习方法和标准化层）相互作用，并为如何更好地适应这种经典方法的现代环境提出建议。我们为我们的建议提供理论支持，并在MLP，经典CNN，具有正常化层，生成性自动编码器和变压器的剩余网络上进行经验验证它们。
The linearised Laplace method for estimating model uncertainty has received renewed attention in the Bayesian deep learning community. The method provides reliable error bars and admits a closed-form expression for the model evidence, allowing for scalable selection of model hyperparameters. In this work, we examine the assumptions behind this method, particularly in conjunction with model selection. We show that these interact poorly with some now-standard tools of deep learning--stochastic approximation methods and normalisation layers--and make recommendations for how to better adapt this classic method to the modern setting. We provide theoretical support for our recommendations and validate them empirically on MLPs, classic CNNs, residual networks with and without normalisation layers, generative autoencoders and transformers.
