自动配置大模型后训练流程，用少于10%算力达到顶尖效果。
Automatic Configuration of LLM Post-Training Pipelines
- 分两阶段：离线学经验，在线用贝叶斯优化快速寻优
- 通过早期信号预测最终性能，每轮评估成本降低
- 适合资源有限但追求高效调参的研究者和工程师
大语言模型后训练流程（结合监督微调与强化学习）在实际算力预算下难以配置：配置空间高维异质，各阶段强耦合，每次端到端评估代价高昂。本文提出 AutoPipe，一种预算感知的两阶段配置选择框架。离线阶段，基于历史运行数据构建数据集相关的学习排序代理，捕捉数据集内偏好，并提供可迁移的配置空间引导。在线阶段，针对新数据集，利用离线引导启动贝叶斯优化，并通过高斯过程残差代理建模数据集特异性偏差。为降低评估成本，每轮试验采用早停策略，并由一个学习得到的预测器将早期训练信号映射为低成本的最终性能代理。在生物医学推理任务上的实验表明，AutoPipe 持续优于仅离线基线，且性能媲美最强在线超参优化基线，同时计算成本不足其10%。
原文摘要 · Abstract (English)
LLM post-training pipelines that combine supervised fine-tuning and reinforcement learning are difficult to configure under realistic compute budgets: the configuration space is high-dimensional and heterogeneous, stages are strongly coupled, and each end-to-end evaluation is expensive. We propose AutoPipe, a budget-aware two-stage framework for configuration selection in LLM post-training. Offline, AutoPipe learns a dataset-conditioned learning-to-rank surrogate from historical runs, capturing within-dataset preferences and providing transferable guidance toward promising regions of the configuration space. Online, for a new dataset, AutoPipe uses the offline guidance to steer Bayesian optimization and models dataset-specific deviations with a Gaussian-process residual surrogate. To reduce evaluation cost, each trial is early-stopped and scored by a learned predictor that maps early training signals to a low-cost proxy for final post-training performance. Experiments on biomedical reasoning tasks show that AutoPipe consistently outperforms offline-only baselines and achieves comparable performance with the strongest online HPO baselines while using less than 10\% of their computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。