论文标题
SGD型方法进行随机优化方法的收敛研究
A convergence study of SGD-type methods for stochastic optimization
论文作者
论文摘要
在本文中，我们首先在更一般的学习率条件和更通用的凸面假设下，从$ l^2 $的意义上重新研究了香草SGD方法的融合，这可以减轻学习率的条件，并且不需要强烈的凸面问题。然后，通过利用Lyapunov功能技术，我们提出了在$ l $ -smooth假设下为凸的动量SGD和Nesterov加速SGD方法的收敛性，该方法在一定程度上扩展了有限的梯度限制。还分析了时间平均SGD的收敛性。
In this paper, we first reinvestigate the convergence of vanilla SGD method in the sense of $L^2$ under more general learning rates conditions and a more general convex assumption, which relieves the conditions on learning rates and do not need the problem to be strongly convex. Then, by taking advantage of the Lyapunov function technique, we present the convergence of the momentum SGD and Nesterov accelerated SGD methods for the convex and non-convex problem under $L$-smooth assumption that extends the bounded gradient limitation to a certain extent. The convergence of time averaged SGD was also analyzed.
