用强化学习动态调整生成参数，让大模型输出更稳定优质。
Adaptive Decoding via Test-Time Policy Learning for Self-Improving Generation
- 将解码过程建模为序列决策，实时学习调整采样策略。
- 在多个数据集上相对基线提升最高达88%，显著改善生成质量。
- 无需重训练模型，适合需要风格灵活的场景和用户定制需求。
解码策略极大影响大语言模型输出质量，但常用的贪婪或固定温度、top-p等启发式方法静态且任务无关，导致跨领域生成质量不稳定或不理想。本文提出一种基于强化学习的解码采样器，将解码视为序列决策过程，在测试时学习轻量级策略动态调整采样参数，同时保持大模型权重冻结。我们在BookSum、arXiv和WikiHow等摘要数据集上，使用Granite-3.3-2B和Qwen-2.5-0.5B进行评估。结果表明，该策略采样器持续优于贪婪和静态基线，在BookSum（Granite）上相对提升高达+88%，在WikiHow（Qwen）上提升+79%。奖励消融实验显示，仅依赖重叠度的目标表现较差，而结合长度、覆盖率、重复性与完整性的结构化奖励项可实现稳定持续改进。这些发现表明，强化学习是实现测试时自适应解码的有效机制，可在不重训练大模型的前提下，实现领域感知与用户可控的生成。
原文摘要 · Abstract (English)
Decoding strategies largely determine the quality of Large Language Model (LLM) outputs, yet widely used heuristics such as greedy or fixed temperature/top-p decoding are static and often task-agnostic, leading to suboptimal or inconsistent generation quality across domains that demand stylistic or structural flexibility. We introduce a reinforcement learning-based decoder sampler that treats decoding as sequential decision-making and learns a lightweight policy to adjust sampling parameters at test-time while keeping LLM weights frozen. We evaluated summarization datasets including BookSum, arXiv, and WikiHow using Granite-3.3-2B and Qwen-2.5-0.5B. Our policy sampler consistently outperforms greedy and static baselines, achieving relative gains of up to +88% (BookSum, Granite) and +79% (WikiHow, Qwen). Reward ablations show that overlap-only objectives underperform compared to composite rewards, while structured shaping terms (length, coverage, repetition, completeness) enable stable and sustained improvements. These findings highlight reinforcement learning as a practical mechanism for test-time adaptation in decoding, enabling domain-aware and user-controllable generation without retraining large models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。