论文标题
daft：蒸馏对面的微调模型，以获得更好的OOD概括
DAFT: Distilling Adversarially Fine-tuned Models for Better OOD Generalization
论文作者
论文摘要
我们考虑了OOD概括的问题，其中的目标是训练在与训练分布不同的测试分布上表现良好的模型。已知深度学习模型在这种转变上脆弱，即使对于略有不同的测试分布，也可能遭受大量精度下降。我们提出了一种基于直觉的新方法 - 愚蠢的方法，即大量丰富特征的对抗性结合应提供鲁棒性。我们的方法仔细地从一位强大的老师那里提炼知识，该知识使用标准培训学习了几个判别特征，同时使用对抗性培训将其结合在一起。对标准的对抗训练程序进行了修改，以产生可以更好地指导学生的教师。我们评估了DAFT在域床框架中的标准基准测试中，并证明DAFT比当前的最新OOD泛化方法取得了重大改进。 daft始终超过表现良好的ERM和蒸馏基线高达6％，对于较小的网络而言，其增益更高。
We consider the problem of OOD generalization, where the goal is to train a model that performs well on test distributions that are different from the training distribution. Deep learning models are known to be fragile to such shifts and can suffer large accuracy drops even for slightly different test distributions. We propose a new method - DAFT - based on the intuition that adversarially robust combination of a large number of rich features should provide OOD robustness. Our method carefully distills the knowledge from a powerful teacher that learns several discriminative features using standard training while combining them using adversarial training. The standard adversarial training procedure is modified to produce teachers which can guide the student better. We evaluate DAFT on standard benchmarks in the DomainBed framework, and demonstrate that DAFT achieves significant improvements over the current state-of-the-art OOD generalization methods. DAFT consistently out-performs well-tuned ERM and distillation baselines by up to 6%, with more pronounced gains for smaller networks.
