FastBO自适应选择计算精度，加速超参与网络结构搜索。
FastBO: Fast HPO and NAS with Adaptive Fidelity Identification
- 根据配置动态选择计算精度，提升效率。
- 在多个数据集上比基线快2~5倍，性能相当。
- 通用框架，可适配任意单精度优化方法。
超参数优化（HPO）和神经网络架构搜索（NAS）是获得顶尖机器学习模型的强大工具，贝叶斯优化（BO）是主流方法。将BO扩展到多精度设置是新兴研究方向，但如何为每个超参数配置确定合适的精度以拟合代理模型仍具挑战。为此，我们提出一种名为FastBO的多精度BO方法，能够自适应地决定每个配置的精度，高效实现优异性能。优势源于对每个配置的‘有效点’和‘饱和点’的新概念。此外，我们的自适应精度识别策略为将任意单精度方法扩展至多精度设置提供了路径，凸显其通用性与适用性。
原文摘要 · Abstract (English)
Hyperparameter optimization (HPO) and neural architecture search (NAS) are powerful in attaining state-of-the-art machine learning models, with Bayesian optimization (BO) standing out as a mainstream method. Extending BO into the multi-fidelity setting has been an emerging research topic, but faces the challenge of determining an appropriate fidelity for each hyperparameter configuration to fit the surrogate model. To tackle the challenge, we propose a multi-fidelity BO method named FastBO, which adaptively decides the fidelity for each configuration and efficiently offers strong performance. The advantages are achieved based on the novel concepts of efficient point and saturation point for each configuration.We also show that our adaptive fidelity identification strategy provides a way to extend any single-fidelity method to the multi-fidelity setting, highlighting its generality and applicability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。