让大模型自动调整推理配置，提升复杂任务表现。
AdaReasoner: Adaptive Reasoning Enables More Flexible Thinking in Large Language Models
- 用强化学习动态优化温度与推理步数等参数
- 在六种模型上均超越基准，知识密集任务增益显著
- 适配不同思考类型，适合需要灵活推理的场景
大语言模型在处理需复杂推理的任务（如数学求解、笑话生成）时，常依赖温度、推理步数等配置。现有提示方法多采用通用固定配置，难以实现任务最优。为此，我们提出AdaReasoner——一种不依赖具体模型的插件，可自动为任意大模型适配推理配置。该方法基于强化学习框架，结合因子化动作空间与针对性探索策略，并利用预训练奖励模型，在仅需少量示例引导下优化策略。理论证明其具有快速收敛和次线性策略差距。在六种不同大模型及多种推理任务上，AdaReasoner持续优于标准基线，保持分布外鲁棒性，并通过定制提示显著提升知识密集型任务表现。
原文摘要 · Abstract (English)
LLMs often need effective configurations, like temperature and reasoning steps, to handle tasks requiring sophisticated reasoning and problem-solving, ranging from joke generation to mathematical reasoning. Existing prompting approaches usually adopt general-purpose, fixed configurations that work 'well enough' across tasks but seldom achieve task-specific optimality. To address this gap, we introduce AdaReasoner, an LLM-agnostic plugin designed for any LLM to automate adaptive reasoning configurations for tasks requiring different types of thinking. AdaReasoner is trained using a reinforcement learning (RL) framework, combining a factorized action space with a targeted exploration strategy, along with a pretrained reward model to optimize the policy model for reasoning configurations with only a few-shot guide. AdaReasoner is backed by theoretical guarantees and experiments of fast convergence and a sublinear policy gap. Across six different LLMs and a variety of reasoning tasks, it consistently outperforms standard baselines, preserves out-of-distribution robustness, and yield gains on knowledge-intensive tasks through tailored prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。