论文标题
深度合奏作为高斯过程近似后部
Deep Ensemble as a Gaussian Process Approximate Posterior
论文作者
论文摘要
深度合奏（DE）是贝叶斯神经网络的有效替代方法，用于深度学习中的不确定性定量。 DE的不确定性通常是由合奏成员之间的功能不一致传达的，例如他们的预测之间的分歧。然而，功能上的不一致源于无法控制的随机性，并且在特定情况下很容易崩溃。为了使DE可靠的不确定性，我们提出了对功能不一致的明确表征的改进，并进一步调整了W.R.T.培训数据和某些先验信念。具体而言，我们描述了合奏成员决定的功能的经验协方差的功能不一致，该功能与平均值一起定义了高斯过程（GP）。然后，由于施加了具体的先验不确定性，我们最大程度地提高了功能性证据下限，以使GP通过DE近似贝叶斯后部指定。通过这种方式，我们将DE与贝叶斯推断有关，以享受可靠的贝叶斯不确定性。此外，我们提供了提高培训效率的策略。我们的方法仅比标准DE的少量增加培训成本，但是比DE及其在各种情况下的变体获得更好的不确定性量化。
Deep Ensemble (DE) is an effective alternative to Bayesian neural networks for uncertainty quantification in deep learning. The uncertainty of DE is usually conveyed by the functional inconsistency among the ensemble members, say, the disagreement among their predictions. Yet, the functional inconsistency stems from unmanageable randomness and may easily collapse in specific cases. To render the uncertainty of DE reliable, we propose a refinement of DE where the functional inconsistency is explicitly characterized, and further tuned w.r.t. the training data and certain priori beliefs. Specifically, we describe the functional inconsistency with the empirical covariance of the functions dictated by ensemble members, which, along with the mean, define a Gaussian process (GP). Then, with specific priori uncertainty imposed, we maximize functional evidence lower bound to make the GP specified by DE approximate the Bayesian posterior. In this way, we relate DE to Bayesian inference to enjoy reliable Bayesian uncertainty. Moreover, we provide strategies to make the training efficient. Our approach consumes only marginally added training cost than the standard DE, but achieves better uncertainty quantification than DE and its variants across diverse scenarios.
