arXiv:2602.08774cs.LGcs.AI2026-02

测试发现机器学习默认参数无法加速贝叶斯优化,随机初始化更优。

Default Machine Learning Hyperparameters Do Not Provide Informative Initialization for Bayesian Optimization

  • 用截断正态分布以库默认值为中心初始化贝叶斯优化
  • 多场景实验均未显著提升收敛速度与最终性能
  • 适合关注超参调优策略的实践者参考

贝叶斯优化(BO)因其在昂贵黑箱函数上的样本效率,已成为超参调优的标准工具。尽管多数BO流程从均匀随机初始化开始,但scikit-learn等主流机器学习库中的默认超参隐含专家知识,可能作为信息性起点加速收敛。这一假设虽直观,却长期缺乏检验。本文通过将BO初始化为以库默认值为中心的截断正态分布点,与均匀随机基线对比。实验涵盖三个BO后端(BoTorch、Optuna、Scikit-Optimize)、三种模型族(随机森林、支持向量机、多层感知机)及五个基准数据集(分类与回归任务)。评估指标包括收敛速度和最终预测质量,统计显著性通过单边二项式检验确定。所有条件下,基于默认值的初始化均无统计显著优势(p值范围0.141至0.908)。敏感性分析显示,虽然更紧的先验方差可提升早期评估表现,但该优势随优化进程消失,最终性能不变。结果表明,默认超参不包含对优化有帮助的方向性信息。建议从业者将超参调优视为模型开发的核心环节,优先采用数据驱动的系统化搜索策略,而非依赖库默认值的启发式假设。

原文摘要 · Abstract (English)

Bayesian Optimization (BO) is a standard tool for hyperparameter tuning thanks to its sample efficiency on expensive black-box functions. While most BO pipelines begin with uniform random initialization, default hyperparameter values shipped with popular ML libraries such as scikit-learn encode implicit expert knowledge and could serve as informative starting points that accelerate convergence. This hypothesis, despite its intuitive appeal, has remained largely unexamined. We formalize the idea by initializing BO with points drawn from truncated Gaussian distributions centered at library defaults and compare the resulting trajectories against a uniform-random baseline. We conduct an extensive empirical evaluation spanning three BO back-ends (BoTorch, Optuna, Scikit-Optimize), three model families (Random Forests, Support Vector Machines, Multilayer Perceptrons), and five benchmark datasets covering classification and regression tasks. Performance is assessed through convergence speed and final predictive quality, and statistical significance is determined via one-sided binomial tests. Across all conditions, default-informed initialization yields no statistically significant advantage over purely random sampling, with p-values ranging from 0.141 to 0.908. A sensitivity analysis on the prior variance confirms that, while tighter concentration around the defaults improves early evaluations, this transient benefit vanishes as optimization progresses, leaving final performance unchanged. Our results provide no evidence that default hyperparameters encode useful directional information for optimization. We therefore recommend that practitioners treat hyperparameter tuning as an integral part of model development and favor principled, data-driven search strategies over heuristic reliance on library defaults.

贝叶斯优化超参调优机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。