论文标题
无模型过多散射的不对称基质分解中的算法正则化
Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization
论文作者
论文摘要
我们研究了自然非凸形公式下的不对称基质分解问题，并具有任意的过多散热性。考虑了无模型设置，对观察到的矩阵的秩或单数值的假设最小，在该矩阵的秩或奇异值中，全局最优值证明是过度拟合的。我们表明，带有小随机初始化的香草梯度下降顺序恢复了观察到的矩阵的主要成分。因此，当配备适当的早期停止时，梯度下降会产生观察到的矩阵的最佳低级别近似，而无需显式正则化。我们提供了近似误差，迭代复杂性，初始化大小和步骤大小之间关系的尖锐表征。我们的复杂性界限几乎不含尺寸，并且取决于对数近似误差，与先前的工作相比，对步骤和初始化的宽大要求明显更大。我们的理论结果为行为梯度下降提供了准确的预测，显示了与数值实验的良好一致性。
We study the asymmetric matrix factorization problem under a natural nonconvex formulation with arbitrary overparametrization. The model-free setting is considered, with minimal assumption on the rank or singular values of the observed matrix, where the global optima provably overfit. We show that vanilla gradient descent with small random initialization sequentially recovers the principal components of the observed matrix. Consequently, when equipped with proper early stopping, gradient descent produces the best low-rank approximation of the observed matrix without explicit regularization. We provide a sharp characterization of the relationship between the approximation error, iteration complexity, initialization size and stepsize. Our complexity bound is almost dimension-free and depends logarithmically on the approximation error, with significantly more lenient requirements on the stepsize and initialization compared to prior work. Our theoretical results provide accurate prediction for the behavior gradient descent, showing good agreement with numerical experiments.
