论文标题
多级培训和贝叶斯优化用于经济高参数优化
Multi-level Training and Bayesian Optimization for Economical Hyperparameter Optimization
论文作者
论文摘要
超参数在许多机器学习方法的表演中起着至关重要的作用。确定其最佳设置或超参数优化（HPO）面临大量超参数以及过度训练时间的困难。在本文中，我们开发了一种有效的方法来减少HPO所需的培训时间。在初始化中，嵌套的拉丁超立方体设计用于选择两种类型的训练的超参数配置，分别是重型训练和轻训练。我们提出了一个截短的加性高斯工艺模型，以使用重型训练产生的准确的性能测量值来校准由轻训练产生的近似性能测量。基于模型，开发了一种基于顺序模型的算法，以生成配置空间的性能配置文件以及找到最佳的算法。我们提出的方法表明，当应用于优化合成示例，支持向量机，完全连接的网络和卷积神经网络时，表明了竞争性能。
Hyperparameters play a critical role in the performances of many machine learning methods. Determining their best settings or Hyperparameter Optimization (HPO) faces difficulties presented by the large number of hyperparameters as well as the excessive training time. In this paper, we develop an effective approach to reducing the total amount of required training time for HPO. In the initialization, the nested Latin hypercube design is used to select hyperparameter configurations for two types of training, which are, respectively, heavy training and light training. We propose a truncated additive Gaussian process model to calibrate approximate performance measurements generated by light training, using accurate performance measurements generated by heavy training. Based on the model, a sequential model-based algorithm is developed to generate the performance profile of the configuration space as well as find optimal ones. Our proposed approach demonstrates competitive performance when applied to optimize synthetic examples, support vector machines, fully connected networks and convolutional neural networks.
