arXiv:2505.18822cs.AIcs.CL2025-05被引 45

让大模型根据题目难易自动调节推理长度,又支持用户手动控制。

AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgeting

  • 基于自评难度动态调整推理长度,避免简单题冗长作答。
  • 在复杂数据集上提升性能同时减少响应长度10%以上,在简单题上压缩90%。
  • 支持用户通过标签精确控制推理深度,适合需要灵活响应的场景。

现代大型推理模型虽具备强大解题能力,但常难以平衡效率与效果,对简单问题也生成过长的推理链。本文提出AdaCtrl框架,实现基于难度感知的自适应推理预算分配和用户显式控制。该框架通过两阶段训练:先冷启动微调使模型具备自我评估难度与调整推理预算的能力,再通过难度感知强化学习优化自适应策略并校准难度判断。设计了显式的长度触发标签作为用户交互接口,实现直观控制。实验证明,相比标准基线(含微调与强化学习),在更复杂的AIME2024和AIME2025数据集上,性能提升的同时响应长度分别减少10.06%和12.14%;在MATH500和GSM8K数据集上,响应长度分别减少62.05%和91.04%。此外,模型支持精准用户控制,可按需生成不同深度的推理结果。

原文摘要 · Abstract (English)

Modern large reasoning models demonstrate impressive problem-solving capabilities by employing sophisticated reasoning strategies. However, they often struggle to balance efficiency and effectiveness, frequently generating unnecessarily lengthy reasoning chains for simple problems. In this work, we propose AdaCtrl, a novel framework to support both difficulty-aware adaptive reasoning budget allocation and explicit user control over reasoning depth. AdaCtrl dynamically adjusts its reasoning length based on self-assessed problem difficulty, while also allowing users to manually control the budget to prioritize either efficiency or effectiveness. This is achieved through a two-stage training pipeline: an initial cold-start fine-tuning phase to instill the ability to self-aware difficulty and adjust reasoning budget, followed by a difficulty-aware reinforcement learning (RL) stage that refines the model's adaptive reasoning strategies and calibrates its difficulty assessments based on its evolving capabilities during online training. To enable intuitive user interaction, we design explicit length-triggered tags that function as a natural interface for budget control. Empirical results show that AdaCtrl adapts reasoning length based on estimated difficulty, compared to the standard training baseline that also incorporates fine-tuning and RL, it yields performance improvements and simultaneously reduces response length by 10.06% and 12.14% on the more challenging AIME2024 and AIME2025 datasets, which require elaborate reasoning, and by 62.05% and 91.04% on the MATH500 and GSM8K datasets, where more concise responses are sufficient. Furthermore, AdaCtrl enables precise user control over the reasoning budget, allowing for tailored responses to meet specific needs.

推理控制自适应预算分配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。