论文标题
渐近最佳的多臂匪徒算法和高参数优化
An Asymptotically Optimal Multi-Armed Bandit Algorithm and Hyperparameter Optimization
论文作者
论文摘要
超参数，神经体系结构或数据增强策略的评估成为高级深度学习中的关键模型选择问题，并具有较大的超级参数搜索空间。在本文中，我们提出了一种在超参数搜索评估的情况下，提出了一种称为子采样（SS）的高效且强大的基于匪徒的算法。它通过观测值的子样本评估了超参数的潜力，从理论上则证明在累积遗憾的标准下是最佳的。我们将SS与贝叶斯优化相结合，并开发出一种称为BOSS的新型超参数优化算法。实证研究验证了我们对SS的理论论点，并证明了Boss在许多应用程序上的出色表现，包括神经体系结构搜索（NAS），数据增强（DA），对象检测（OD）和强化学习（RL）。
The evaluation of hyperparameters, neural architectures, or data augmentation policies becomes a critical model selection problem in advanced deep learning with a large hyperparameter search space. In this paper, we propose an efficient and robust bandit-based algorithm called Sub-Sampling (SS) in the scenario of hyperparameter search evaluation. It evaluates the potential of hyperparameters by the sub-samples of observations and is theoretically proved to be optimal under the criterion of cumulative regret. We further combine SS with Bayesian Optimization and develop a novel hyperparameter optimization algorithm called BOSS. Empirical studies validate our theoretical arguments of SS and demonstrate the superior performance of BOSS on a number of applications, including Neural Architecture Search (NAS), Data Augmentation (DA), Object Detection (OD), and Reinforcement Learning (RL).
