用轻量在线方法提升超参数优化效率，性能提升0.2%至7%
Sequential Policy Gradient for Adaptive Hyperparameter Optimization
- 通过临时模块实现单次前向传播生成动作轨迹
- 在5个数据集上实现0.2%~7%的性能提升
- 适合需要高效调参的工业级模型部署场景
强化学习对神经网络架构搜索和超参数优化至关重要，但传统方法因时间和计算成本过高而难以普及。受DeepSeek-V3多标记预测架构启发，本文提出序列策略梯度建模（SPG），一种轻量级在线超参数优化的新轨迹生成范式。与传统策略梯度方法不同，SPG通过添加临时模块，使基础模型能在单次前向传播中生成带填充的状态-动作轨迹。实验表明，模型在原始数据集上使用SPG重训练后性能提升，且优于标准迁移微调。我们在计算机视觉（ImageNet、COCO）、自然语言处理（GLUE、SQuAD）和音频（SUPERB）五个数据集上评估，验证了SPG的工业适用性。该方法在主流模型上均表现稳定，性能提升达+0.2%~7%，且计算开销极低。完整可复现代码与预训练模型已开源：https://huggingface.co/UniversalAlgorithmic/SPG。
原文摘要 · Abstract (English)
Reinforcement learning is essential for neural architecture search and hyperparameter optimization, but the conventional approaches impede widespread use due to prohibitive time and computational costs. Inspired by DeepSeek-V3 multi-token prediction architecture, we propose Sequential Policy Gradient modeling (SPG), a novel trajectory generation paradigm for lightweight online hyperparameter optimization. In contrast to conventional policy gradient methods, SPG extends the base model with temporary modules, enabling it to generate state-action (padded) trajectories in a single forward pass. Our experiments demonstrate that models gain performance when retrained with SPG on their original datasets and also outperform standard transfer fine-tuning. We evaluate on five datasets spanning computer vision (ImageNet, COCO), natural language processing (GLUE, SQuAD), and audio (SUPERB) to assess the industrial applicability of SPG. The proposed method demonstrates consistent improvements across widely adopted models, achieving performance gains of $+0.2\sim7\%$, with significantly low computational costs. Fully reproducible code and pre-trained models: https://huggingface.co/UniversalAlgorithmic/SPG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。