论文标题
非平滑非凸最小化的亚级别采样
Subgradient sampling for nonsmooth nonconvex minimization
论文作者
论文摘要
非平滑非概要问题的风险最小化自然会导致一阶采样，或者滥用术语来到随机下降下降。我们在路径差异的情况下建立了该方法的收敛性，并在其他几何假设下描述了更精确的结果。我们从Ermoliev和Norkin [Cybern恢复并改善结果。系统。 Anal。，34（1998），第196--215页]通过使用不同的方法：保守的演算和ODE方法。在可定义的情况下，我们表明，一阶次级速度采样避免了具有概率的人造关键点，并且还适用于基于反向流行的Oracle，在深度学习中适用于大量的风险最小化问题。作为我们方法的副产品，我们获得了独立利益集成的几个结果，例如保守衍生物和积分的互换结果，或设置值参数化积分的确定性。
Risk minimization for nonsmooth nonconvex problems naturally leads to first-order sampling or, by an abuse of terminology, to stochastic subgradient descent. We establish the convergence of this method in the path-differentiable case and describe more precise results under additional geometric assumptions. We recover and improve results from Ermoliev and Norkin [Cybern. Syst. Anal., 34 (1998), pp. 196--215] by using a different approach: conservative calculus and the ODE method. In the definable case, we show that first-order subgradient sampling avoids artificial critical points with probability one and applies moreover to a large range of risk minimization problems in deep learning, based on the backpropagation oracle. As byproducts of our approach, we obtain several results on integration of independent interest, such as an interchange result for conservative derivatives and integrals or the definability of set-valued parameterized integrals.
