论文标题
单独有条件的单个互助信息绑定了概括误差
Individually Conditional Individual Mutual Information Bound on Generalization Error
论文作者
论文摘要
我们基于Bu等人的误差分解技术的组合，提出了一种基于概括误差的新信息理论。以及Steinke和Zakynthinou的条件互信息（CMI）构建。在先前的工作中，Haghifam等。提出了一种不同的结合，结合了上述两种技术，我们称这是条件个体相互信息（CIMI）结合的。但是，在简单的高斯环境中，CMI和CIMI界限的秩序比Bu等人的秩序差。这种观察促使我们提出了新的界限，这通过减少条件互信息中的条件术语来克服了这个问题。在建立这种结合的过程中，建立了有条件的脱钩引理，这也导致这些信息理论界限之间有意义的二分法和比较。
We propose a new information-theoretic bound on generalization error based on a combination of the error decomposition technique of Bu et al. and the conditional mutual information (CMI) construction of Steinke and Zakynthinou. In a previous work, Haghifam et al. proposed a different bound combining the two aforementioned techniques, which we refer to as the conditional individual mutual information (CIMI) bound. However, in a simple Gaussian setting, both the CMI and the CIMI bounds are order-wise worse than that by Bu et al.. This observation motivated us to propose the new bound, which overcomes this issue by reducing the conditioning terms in the conditional mutual information. In the process of establishing this bound, a conditional decoupling lemma is established, which also leads to a meaningful dichotomy and comparison among these information-theoretic bounds.
