论文标题
对数据不足学习的均衡性和不变性偏见
Equivariance and Invariance Inductive Bias for Learning from Insufficient Data
论文作者
论文摘要
我们有兴趣从数据不足中学习强大的模型，而无需任何外部预训练的检查点。首先，与足够的数据相比，我们展示了为什么数据不足会使模型更容易偏向于通常不同于测试的有限培训环境。例如，如果所有训练天鹅样本都是“白色”，则该模型可能会错误地使用“白色”环境来代表内在的天鹅。然后，我们证明均衡感应偏差可以保留类功能，而不变性偏见可以消除环境功能，而将类功能概括为测试中任何环境变化。为了将它们强加于学习，我们证明可以部署任何基于基于对比的自我监督的特征学习方法；对于不变性，我们提出了一个范围的不变风险最小化（IRM），该风险最小化（IRM）有效地应对传统IRM中缺少环境注释的挑战。对现实世界基准（Vipriors，imagenet100和Nico）的最新实验结果验证了均衡性和不变性在数据效率学习中的巨大潜力。该代码可从https://github.com/wangt-cn/eqinv获得
We are interested in learning robust models from insufficient data, without the need for any externally pre-trained checkpoints. First, compared to sufficient data, we show why insufficient data renders the model more easily biased to the limited training environments that are usually different from testing. For example, if all the training swan samples are "white", the model may wrongly use the "white" environment to represent the intrinsic class swan. Then, we justify that equivariance inductive bias can retain the class feature while invariance inductive bias can remove the environmental feature, leaving the class feature that generalizes to any environmental changes in testing. To impose them on learning, for equivariance, we demonstrate that any off-the-shelf contrastive-based self-supervised feature learning method can be deployed; for invariance, we propose a class-wise invariant risk minimization (IRM) that efficiently tackles the challenge of missing environmental annotation in conventional IRM. State-of-the-art experimental results on real-world benchmarks (VIPriors, ImageNet100 and NICO) validate the great potential of equivariance and invariance in data-efficient learning. The code is available at https://github.com/Wangt-CN/EqInv
