用强化学习+Transformer优化超参数，效率更高更稳定。
GRPOformer: Advancing Hyperparameter Optimization via Group Relative Policy Optimization
- 用Transformer生成新超参数配置，结合GRPO快速学习优化策略。
- 在OpenML上优于基线方法，跨任务表现一致领先。
- 适合需要高效调参的机器学习研究者和工程师。
超参数优化（HPO）对提升模型性能至关重要。基于Transformer的HPO方法展现出巨大潜力，但现有方法严重依赖大规模历史优化轨迹，且缺乏有效的强化学习（RL）技术，限制了其效率与性能提升。受大语言模型中群体相对策略优化（GRPO）成功的启发，我们提出GRPOformer——一种将强化学习与Transformer结合的新型超参数优化框架。在GRPOformer中，Transformer从历史优化轨迹中生成新的超参数配置，而GRPO则实现从零开始的快速轨迹构建与优化策略学习。此外，我们引入策略波动正则化（PCR）以增强GRPO训练的稳定性。在OpenML数据集上的实验结果表明，GRPOformer在多样化任务中均持续优于基线方法，为强化学习在HPO中的应用提供了新思路。
原文摘要 · Abstract (English)
Hyperparameter optimization (HPO) plays a critical role in improving model performance. Transformer-based HPO methods have shown great potential; however, existing approaches rely heavily on large-scale historical optimization trajectories and lack effective reinforcement learning (RL) techniques, thereby limiting their efficiency and performance improvements. Inspired by the success of Group Relative Policy Optimization (GRPO) in large language models (LLMs), we propose GRPOformer -- a novel hyperparameter optimization framework that integrates reinforcement learning (RL) with Transformers. In GRPOformer, Transformers are employed to generate new hyperparameter configurations from historical optimization trajectories, while GRPO enables rapid trajectory construction and optimization strategy learning from scratch. Moreover, we introduce Policy Churn Regularization (PCR) to enhance the stability of GRPO training. Experimental results on OpenML demonstrate that GRPOformer consistently outperforms baseline methods across diverse tasks, offering new insights into the application of RL for HPO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。