论文标题
通过神经网络积极学习：非参数统计的见解
Active Learning with Neural Networks: Insights from Nonparametric Statistics
论文作者
论文摘要
深神经网络具有很大的代表力，但通常需要大量的培训示例。这激发了深厚的积极学习方法，可以显着减少标记的培训数据的量。最近在文献中报道了深度积极学习的经验成功，但是，严格的标签复杂性保证了深度积极学习仍然难以捉摸。这构成了理论和实践之间的显着差距。本文通过提供第一个近乎最佳的标签复杂性来解决这一差距，以确保深入积极学习。关键的见解是从非参数分类的角度研究深入的积极学习。在标准的低噪声条件下，我们表明，使用神经网络的主动学习可以证明达到最小标签的复杂性，直到分歧系数和其他对数项。当配备弃权选项时，我们进一步开发了一种有效的深度积极学习算法，该算法可实现$ \ mathsf {polylog}（\ frac {1}ε）$标签复杂性，而没有任何低噪声假设。除了经常研究的Sobolev/Hölder空间外，我们还提供了我们的结果的扩展，并在radon $ \ Mathsf {bv}^2 $空间中开发了标签复杂性，以确保学习，这些空间最近被提出为与神经网络相关的自然功能空间。
Deep neural networks have great representation power, but typically require large numbers of training examples. This motivates deep active learning methods that can significantly reduce the amount of labeled training data. Empirical successes of deep active learning have been recently reported in the literature, however, rigorous label complexity guarantees of deep active learning have remained elusive. This constitutes a significant gap between theory and practice. This paper tackles this gap by providing the first near-optimal label complexity guarantees for deep active learning. The key insight is to study deep active learning from the nonparametric classification perspective. Under standard low noise conditions, we show that active learning with neural networks can provably achieve the minimax label complexity, up to disagreement coefficient and other logarithmic terms. When equipped with an abstention option, we further develop an efficient deep active learning algorithm that achieves $\mathsf{polylog}(\frac{1}ε)$ label complexity, without any low noise assumptions. We also provide extensions of our results beyond the commonly studied Sobolev/Hölder spaces and develop label complexity guarantees for learning in Radon $\mathsf{BV}^2$ spaces, which have recently been proposed as natural function spaces associated with neural networks.
