论文标题
minimax后悔优化了在分销转移中的强大机器学习
Minimax Regret Optimization for Robust Machine Learning under Distribution Shift
论文作者
论文摘要
在本文中，我们考虑学习方案，其中在未知的测试分布中评估了学习模型，该测试分布可能与训练分布不同（即分配变化）。学习者可以访问一个重量功能家族，以便测试分布是对其中一个功能下的训练分布的重新加权，该设置通常以分布强大的优化（DRO）的名称进行研究。我们考虑在经典学习理论设置中得出后悔界限的问题，并要求所产生的遗憾界限统一地为所有潜在的测试分布而言。我们表明，DRO配方并不能保证在分配转移中统一的遗憾。相反，我们提出了一种称为Minimax遗憾优化（MRO）的替代方法，并表明在适当的条件下，此方法在所有测试分布中都统一遗憾。当测试分布与培训数据的相似性异质时，我们还适应了我们的技术，以获得更强的保证。鉴于在当前的强大机器学习方法中对最坏情况的风险进行了广泛的优化，我们认为MRO可以是解决分销转移方案的强大替代方法。
In this paper, we consider learning scenarios where the learned model is evaluated under an unknown test distribution which potentially differs from the training distribution (i.e. distribution shift). The learner has access to a family of weight functions such that the test distribution is a reweighting of the training distribution under one of these functions, a setting typically studied under the name of Distributionally Robust Optimization (DRO). We consider the problem of deriving regret bounds in the classical learning theory setting, and require that the resulting regret bounds hold uniformly for all potential test distributions. We show that the DRO formulation does not guarantee uniformly small regret under distribution shift. We instead propose an alternative method called Minimax Regret Optimization (MRO), and show that under suitable conditions this method achieves uniformly low regret across all test distributions. We also adapt our technique to have stronger guarantees when the test distributions are heterogeneous in their similarity to the training data. Given the widespead optimization of worst case risks in current approaches to robust machine learning, we believe that MRO can be a strong alternative to address distribution shift scenarios.
