论文标题
高维数据集的结合稀疏输入层次网络
Ensembled sparse-input hierarchical networks for high-dimensional datasets
论文作者
论文摘要
神经网络在预测中的使用有限的高维数据的使用量有限，因为它们倾向于过度贴合，并且需要比现有的现成机器学习方法更多的超级参数。通过对网络体系结构和培训程序进行小修改，我们表明密集的神经网络可以成为这些设置中的实用数据分析工具。提出的方法是通过平均稀疏输入层次网络（更轻松的网络）进行的合奏，通过仅调谐两个L1-Penalty参数来适当地修复网络结构，一个参数可以控制输入稀数，另一个控制隐藏层和节点的数量。如果无关协变量仅与响应微弱相关，则该方法从真实支持中选择变量；否则，它会表现出分组效应，其中以相似的速率选择了密切相关的协变量。在具有不同大小，更轻松的网络体系结构的现实世界数据集的集合中，与平均现成的方法相比，预测准确性更高。
Neural networks have seen limited use in prediction for high-dimensional data with small sample sizes, because they tend to overfit and require tuning many more hyperparameters than existing off-the-shelf machine learning methods. With small modifications to the network architecture and training procedure, we show that dense neural networks can be a practical data analysis tool in these settings. The proposed method, Ensemble by Averaging Sparse-Input Hierarchical networks (EASIER-net), appropriately prunes the network structure by tuning only two L1-penalty parameters, one that controls the input sparsity and another that controls the number of hidden layers and nodes. The method selects variables from the true support if the irrelevant covariates are only weakly correlated with the response; otherwise, it exhibits a grouping effect, where strongly correlated covariates are selected at similar rates. On a collection of real-world datasets with different sizes, EASIER-net selected network architectures in a data-adaptive manner and achieved higher prediction accuracy than off-the-shelf methods on average.
