论文标题

关于具有控制同步步骤的不精确梯度下降的收敛性

On the Convergence of Inexact Gradient Descent with Controlled Synchronization Steps

论文作者

Ranaweera, Sandushan, Weeraddana, Chathuranga, Dharmawansa, Prathapasinghe, Fischione, Carlo

论文摘要

我们开发了一种类似梯度的算法,以最大程度地减少基于同行互连网络协调的同伴目标函数的总和。该协调允许两个阶段:第一个是构成一个可能存在错误的梯度,以更新每个对等的本地复制决策变量,第二个用于同步本地复制品的无错误平均。与许多相关算法不同,我们算法中允许的误差只要有界限,就可以涵盖多种不符合性。此外,我们没有为目标函数施加任何梯度界限条件。此外,第二阶段不像许多相关算法那样定期进行。取而代之的是,设计了一个本地可验证的标准,以在第二阶段动态触发点对点协调,因此可以大大减少无错误的平均昂贵通信开销。最后,在轻度条件下建立算法的收敛性。

We develop a gradient-like algorithm to minimize a sum of peer objective functions based on coordination through a peer interconnection network. The coordination admits two stages: the first is to constitute a gradient, possibly with errors, for updating locally replicated decision variables at each peer and the second is used for error-free averaging for synchronizing local replicas. Unlike many related algorithms, the errors permitted in our algorithm can cover a wide range of inexactnesses, as long as they are bounded. Moreover, we do not impose any gradient boundedness conditions for the objective functions. Furthermore, the second stage is not conducted in a periodic manner, like many related algorithms. Instead, a locally verifiable criterion is devised to dynamically trigger the peer-to-peer coordination at the second stage, so that expensive communication overhead for error-free averaging can significantly be reduced. Finally, the convergence of the algorithm is established under mild conditions.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源