论文标题
具有偏深的学习的差异私人框架,具有共透性损失功能
A Differentially Private Framework for Deep Learning with Convexified Loss Functions
论文作者
论文摘要
差异隐私(DP)已应用于深度学习,以保护基础培训集的隐私。现有的DP实践分为三类 - 客观扰动,梯度扰动和输出扰动。他们遭受三个主要问题。首先,目标功能的条件限制了一般深度学习任务中的客观扰动。其次,由于每个时期内注入过度注入的噪声,梯度扰动并不能实现令人满意的隐私 - 实用性权衡。第三,由于训练有素的模型参数作为噪声量表参数,因此无法保证输出扰动方法的高效用。为了解决这些问题,我们分析了模型参数的全局灵敏度的更严格的上限。在基于这种全局敏感性的黑框设置下,为了控制整体噪声注入,我们通过将DP噪声注入基线非私有神经网络的输出层的随机采样神经元(通过指数机制),提出了一个新型的输出扰动框架,该噪声具有培训的基线非私有神经网络,该神经网络训练有良好的损失功能。我们从经验上比较了与基线非私有模型的准确损失以及针对黑盒会员推理(MI)攻击,我们的框架和开放源代码差异化私有随机梯度下降(DP-SGD)在六个常用的现实世界中的六个常用现实世界数据集对黑盒会员推理(MI)攻击的隐私损失以及针对黑盒会员推理(MI)攻击的隐私泄漏衡量的。实验评估表明,当基线模型在MI攻击下可观察到的隐私泄漏时,我们的框架比现有的DP-SGD实施实现了更好的隐私性 - 实用性权衡,鉴于总体隐私预算$ε\ε\ leq 1 $,供大量查询。
Differential privacy (DP) has been applied in deep learning for preserving privacy of the underlying training sets. Existing DP practice falls into three categories - objective perturbation, gradient perturbation and output perturbation. They suffer from three main problems. First, conditions on objective functions limit objective perturbation in general deep learning tasks. Second, gradient perturbation does not achieve a satisfactory privacy-utility trade-off due to over-injected noise in each epoch. Third, high utility of the output perturbation method is not guaranteed because of the loose upper bound on the global sensitivity of the trained model parameters as the noise scale parameter. To address these problems, we analyse a tighter upper bound on the global sensitivity of the model parameters. Under a black-box setting, based on this global sensitivity, to control the overall noise injection, we propose a novel output perturbation framework by injecting DP noise into a randomly sampled neuron (via the exponential mechanism) at the output layer of a baseline non-private neural network trained with a convexified loss function. We empirically compare the privacy-utility trade-off, measured by accuracy loss to baseline non-private models and the privacy leakage against black-box membership inference (MI) attacks, between our framework and the open-source differentially private stochastic gradient descent (DP-SGD) approaches on six commonly used real-world datasets. The experimental evaluations show that, when the baseline models have observable privacy leakage under MI attacks, our framework achieves a better privacy-utility trade-off than existing DP-SGD implementations, given an overall privacy budget $ε\leq 1$ for a large number of queries.