论文标题
私人非凸的动量聚合
Momentum Aggregation for Private Non-convex ERM
论文作者
论文摘要
我们在平稳的$ d $维度目标上介绍了新的算法和融合保证，以保证具有隐私性的非凸经验风险最小化（ERM）。我们在平滑目标上开发了对随机梯度下降的提高灵敏度分析，该目标利用了不同时期中例子的复发。通过将这种新方法与最新动量分析与私人聚合技术相结合，我们提供了一种$（ε，δ）$ - 差异私人算法，可以找到Norm $ \ tilde o \ left的梯度$ o \ left（\ frac {n^{7/3}ε^{4/3}}} {d^{2/3}}} \ right）$渐变评估，改善了以前的最佳梯度限制的$ \ tilde o \ left（\ frac {\ frac {d^d^{d^{1/4} {1/4}} {1/4}} {
We introduce new algorithms and convergence guarantees for privacy-preserving non-convex Empirical Risk Minimization (ERM) on smooth $d$-dimensional objectives. We develop an improved sensitivity analysis of stochastic gradient descent on smooth objectives that exploits the recurrence of examples in different epochs. By combining this new approach with recent analysis of momentum with private aggregation techniques, we provide an $(ε,δ)$-differential private algorithm that finds a gradient of norm $\tilde O\left(\frac{d^{1/3}}{(εN)^{2/3}}\right)$ in $O\left(\frac{N^{7/3}ε^{4/3}}{d^{2/3}}\right)$ gradient evaluations, improving the previous best gradient bound of $\tilde O\left(\frac{d^{1/4}}{\sqrt{εN}}\right)$.
