论文标题
联合平均Langevin动力学：迈向统一理论和新算法
Federated Averaging Langevin Dynamics: Toward a unified theory and new algorithms
论文作者
论文摘要
本文重点介绍了联合学习环境（FL）中的贝叶斯推论。尽管已经提出了几种分布的MCMC算法，但很少有人考虑FL的特定局限性，例如通信瓶颈和统计异质性。最近，引入了联邦平均兰格文动力学（FALD），这将联邦平均算法扩展到贝叶斯推论。我们在瓦尔德（Fald）的全球后部距离上获得了一种新型的紧密非质合上限。这一界限强调了统计异质性的影响，这会导致局部更新的漂移，从而对收敛产生负面影响。我们提出了一种使用控制变体来纠正客户端漂移的新算法vr-fald*。我们建立了非反应界限，表明VR-Fald*不受统计异质性的影响。最后，我们在贝叶斯推断的几个FL基准上说明了结果。
This paper focuses on Bayesian inference in a federated learning context (FL). While several distributed MCMC algorithms have been proposed, few consider the specific limitations of FL such as communication bottlenecks and statistical heterogeneity. Recently, Federated Averaging Langevin Dynamics (FALD) was introduced, which extends the Federated Averaging algorithm to Bayesian inference. We obtain a novel tight non-asymptotic upper bound on the Wasserstein distance to the global posterior for FALD. This bound highlights the effects of statistical heterogeneity, which causes a drift in the local updates that negatively impacts convergence. We propose a new algorithm VR-FALD* that uses control variates to correct the client drift. We establish non-asymptotic bounds showing that VR-FALD* is not affected by statistical heterogeneity. Finally, we illustrate our results on several FL benchmarks for Bayesian inference.
