论文标题
关于深豪斯过程的变异推断中的信噪比问题
On Signal-to-Noise Ratio Issues in Variational Inference for Deep Gaussian Processes
论文作者
论文摘要
我们表明，具有重要性加权变量推理的训练深高斯过程（DGP）中使用的梯度估计值容易受到信噪比（SNR）问题的影响。具体而言，我们在理论上和通过广泛的经验评估都表明，随着重要性样本的增加，潜在变量变量参数的梯度估计值降低。结果，如果重要性的数量太大，这些梯度估计会降低纯噪声。为了解决这种病理，我们展示了如何将最初用于培训变种自动编码器提议的双重重新聚集化梯度估计器适应DGP设置，并且所得的估计器完全可以完全解决SNR问题，从而提供了更可靠的培训。最后，我们证明我们的修复可以导致DGP模型的预测性能的一致改进。
We show that the gradient estimates used in training Deep Gaussian Processes (DGPs) with importance-weighted variational inference are susceptible to signal-to-noise ratio (SNR) issues. Specifically, we show both theoretically and via an extensive empirical evaluation that the SNR of the gradient estimates for the latent variable's variational parameters decreases as the number of importance samples increases. As a result, these gradient estimates degrade to pure noise if the number of importance samples is too large. To address this pathology, we show how doubly reparameterized gradient estimators, originally proposed for training variational autoencoders, can be adapted to the DGP setting and that the resultant estimators completely remedy the SNR issue, thereby providing more reliable training. Finally, we demonstrate that our fix can lead to consistent improvements in the predictive performance of DGP models.
