论文标题
通过确定性PAC-BAYES的梯度下降下的概括
Generalisation under gradient descent via deterministic PAC-Bayes
论文作者
论文摘要
我们为训练有梯度下降方法或连续梯度流训练的模型建立了崩解的Pac-bayesian泛化界限。与Pac-Bayesian环境中的标准实践相反，我们的结果适用于确定性的优化算法，而无需任何脱离随机的步骤。我们的边界是完全可计算的，具体取决于初始分布的密度和轨迹上训练目标的Hessian。我们表明，我们的框架可以应用于各种迭代优化算法，包括随机梯度下降（SGD），基于动量的方案和抑制的哈密顿动力学。
We establish disintegrated PAC-Bayesian generalisation bounds for models trained with gradient descent methods or continuous gradient flows. Contrary to standard practice in the PAC-Bayesian setting, our result applies to optimisation algorithms that are deterministic, without requiring any de-randomisation step. Our bounds are fully computable, depending on the density of the initial distribution and the Hessian of the training objective over the trajectory. We show that our framework can be applied to a variety of iterative optimisation algorithms, including stochastic gradient descent (SGD), momentum-based schemes, and damped Hamiltonian dynamics.
