论文标题
优化衍生的学习，对训练和超训练的基本收敛分析
Optimization-Derived Learning with Essential Convergence Analysis of Training and Hyper-training
论文作者
论文摘要
最近，优化衍生的学习（ODL）吸引了学习和视觉领域的关注，该学习和视觉领域从优化的角度设计了学习模型。但是，以前的ODL方法将训练和超训练程序视为两个分离的阶段，这意味着在训练过程中必须固定超训练变量，因此也不可能同时获得训练和超训练变量的收敛性。在这项工作中，我们将基于定点迭代的广义Krasnoselskii-Mann（GKM）计划设计为我们的基本ODL模块，该模块将现有的ODL方法统一为特殊情况。在GKM方案下，构建了二聚体元优化（BMO）算法框架，以共同解决最佳训练和超训练变量。我们严格地证明了训练定点迭代的基本关节融合以及优化超培训的超训练的过程，无论是在近似质量方面还是在固定分析上。实验证明了BMO在稀疏编码和现实世界应用中具有竞争性能的效率，例如图像反卷积和降雨的删除。
Recently, Optimization-Derived Learning (ODL) has attracted attention from learning and vision areas, which designs learning models from the perspective of optimization. However, previous ODL approaches regard the training and hyper-training procedures as two separated stages, meaning that the hyper-training variables have to be fixed during the training process, and thus it is also impossible to simultaneously obtain the convergence of training and hyper-training variables. In this work, we design a Generalized Krasnoselskii-Mann (GKM) scheme based on fixed-point iterations as our fundamental ODL module, which unifies existing ODL methods as special cases. Under the GKM scheme, a Bilevel Meta Optimization (BMO) algorithmic framework is constructed to solve the optimal training and hyper-training variables together. We rigorously prove the essential joint convergence of the fixed-point iteration for training and the process of optimizing hyper-parameters for hyper-training, both on the approximation quality, and on the stationary analysis. Experiments demonstrate the efficiency of BMO with competitive performance on sparse coding and real-world applications such as image deconvolution and rain streak removal.
