论文标题
非凸优化的动量加速自适应立方正规化方法
A Momentum Accelerated Adaptive Cubic Regularization Method for Nonconvex Optimization
论文作者
论文摘要
立方正则化方法（CR）及其自适应版本（ARC）是牛顿型的流行方法，用于解决无约束的非凸优化问题，这是由于其在轻度条件下与局部最小值的全球收敛性。本文的主要目的是开发一种动量加速自适应立方体化方法（ARCM），以提高收敛性能。通过正确选择动量步长的大小，我们还可以在\ kl属性下保证ARCM的全局收敛性和本地收敛性。当迭代程序中采用低计算成本的不精确求解器时，也可以建立这种全球和局部收敛。据报道，非凸面逻辑回归和鲁棒线性回归模型的数值结果表明，所提出的ARCM显着胜过最先进的立方正则方法（例如CR，基于动量的CR，ARC）和信任区域方法。特别是，在实验中，ARCM所需的迭代次数小于10 \％至50 \％。
The cubic regularization method (CR) and its adaptive version (ARC) are popular Newton-type methods in solving unconstrained non-convex optimization problems, due to its global convergence to local minima under mild conditions. The main aim of this paper is to develop a momentum-accelerated adaptive cubic regularization method (ARCm) to improve the convergent performance. With the proper choice of momentum step size, we show the global convergence of ARCm and the local convergence can also be guaranteed under the \KL property. Such global and local convergence can also be established when inexact solvers with low computational costs are employed in the iteration procedure. Numerical results for non-convex logistic regression and robust linear regression models are reported to demonstrate that the proposed ARCm significantly outperforms state-of-the-art cubic regularization methods (e.g., CR, momentum-based CR, ARC) and the trust region method. In particular, the number of iterations required by ARCm is less than 10\% to 50\% required by the most competitive method (ARC) in the experiments.
