arXiv:2410.08920cs.LGcs.AI2024-10被引 6

提出高效方法评估CNN超参数重要性,帮研究者聚焦关键调参项。

Efficient Hyperparameter Importance Assessment for CNNs

  • 用N-RReliefF算法量化超参数重要性,避免盲目搜索。
  • 实验训练超1万模型,验证学习率、层数等前五重要超参数。
  • 适合想省时省力调优的深度学习实践者参考。

超参数选择是机器学习流程中的关键环节,深刻影响模型的鲁棒性、稳定性与泛化能力。由于神经网络超参数空间复杂且受计算资源与时间限制,全面优化所有超参数不现实。在此背景下,利用超参数重要性评估(HIA)可有效缩小搜索范围,帮助从业者将优化精力集中于对性能影响最大的超参数上,节省时间和资源。本文提出N-RReliefF算法,量化卷积神经网络(CNN)中部分超参数的重要性,为深度学习领域应用HIA方法奠定基础。通过在十个主流图像分类数据集上训练超过一万种CNN模型,构建了包含超参数配置及其性能指标的综合数据集。结果表明,在所考察的超参数中,对CNN模型影响最显著的前五位依次为卷积层数量、学习率、丢弃率、优化器类型和训练轮数。

原文摘要 · Abstract (English)

Hyperparameter selection is an essential aspect of the machine learning pipeline, profoundly impacting models' robustness, stability, and generalization capabilities. Given the complex hyperparameter spaces associated with Neural Networks and the constraints of computational resources and time, optimizing all hyperparameters becomes impractical. In this context, leveraging hyperparameter importance assessment (HIA) can provide valuable guidance by narrowing down the search space. This enables machine learning practitioners to focus their optimization efforts on the hyperparameters with the most significant impact on model performance while conserving time and resources. This paper aims to quantify the importance weights of some hyperparameters in Convolutional Neural Networks (CNNs) with an algorithm called N-RReliefF, laying the groundwork for applying HIA methodologies in the Deep Learning field. We conduct an extensive study by training over ten thousand CNN models across ten popular image classification datasets, thereby acquiring a comprehensive dataset containing hyperparameter configuration instances and their corresponding performance metrics. It is demonstrated that among the investigated hyperparameters, the top five important hyperparameters of the CNN model are the number of convolutional layers, learning rate, dropout rate, optimizer and epoch.

超参数优化卷积神经网络重要性评估机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。