自动调整超参更新频率，让训练更省力高效。
Iterated Population Based Training with Task-Agnostic Restarts
- 通过任务无关重启机制动态优化超参更新间隔。
- 8个任务上平均性能超越5种现有算法，无需额外计算预算。
- 适合希望减少调参负担的研究者和工程团队。
超参数优化（HPO）可减轻神经网络超参数调优的负担。基于群体进化训练（PBT）的算法因能在权重优化每几步后动态调整超参数而高效。最新研究表明，各PBT变体中，超参数更新间隔这一元超参数对性能影响显著，但目前缺乏有效设置方法。本文提出迭代式群体进化训练（IPBT），通过重用权重信息的任务无关重启机制，结合随时间变化的贝叶斯优化，自动调节该超参数。在8个图像分类与强化学习任务上的评估显示，平均而言，IPBT性能匹配或超越5种现有PBT变体及其他HPO算法（随机搜索、ASHA、SMAC3），且无需增加计算预算或修改自身超参数。代码开源：https://github.com/AwesomeLemon/IPBT。
原文摘要 · Abstract (English)
Hyperparameter Optimization (HPO) can lift the burden of tuning hyperparameters (HPs) of neural networks. HPO algorithms from the Population Based Training (PBT) family are efficient thanks to dynamically adjusting HPs every few steps of the weight optimization. Recent results indicate that the number of steps between HP updates is an important meta-HP of all PBT variants that can substantially affect their performance. Yet, no method or intuition is available for efficiently setting its value. We introduce Iterated Population Based Training (IPBT), a novel PBT variant that automatically adjusts this HP via restarts that reuse weight information in a task-agnostic way and leverage time-varying Bayesian optimization to reinitialize HPs. Evaluation on 8 image classification and reinforcement learning tasks shows that, on average, our algorithm matches or outperforms 5 previous PBT variants and other HPO algorithms (random search, ASHA, SMAC3), without requiring a budget increase or any changes to its HPs. The source code is available at https://github.com/AwesomeLemon/IPBT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。