arXiv:2508.18773cs.CL2025-08被引 8

ThinkDial让大模型像调旋钮一样控制推理强度,兼顾效率与性能。

ThinkDial: An Open Recipe for Controlling Reasoning Effort in Large Language Models

  • 通过端到端训练,将推理模式控制嵌入学习过程,实现三种可调模式。
  • 低模式节省75%令牌,性能下降小于15%;中模式节省50%令牌,性能降不到10%。
  • 开源框架,适合需要灵活控制推理成本的部署场景。

大型语言模型(LLMs)在链式思维推理方面展现出卓越的问题解决能力,但其计算资源消耗难以控制,制约了实际应用。尽管像OpenAI的gpt-oss系列这类专有系统已引入离散操作模式实现直观的推理控制,但开源社区尚未达成类似效果。本文提出ThinkDial,首个开源端到端框架,成功复现gpt-oss式的可控推理。系统支持三种推理模式:高模式(完整推理能力)、中模式(减少50%生成令牌,性能下降<10%)、低模式(减少75%生成令牌,性能下降<15%)。通过预算模式监督微调和两阶段预算感知强化学习(含自适应奖励设计),将控制机制贯穿整个推理流程。大量实验表明,ThinkDial在保持性能阈值的同时实现明确的响应长度压缩,并在分布外任务上表现出强泛化能力。

原文摘要 · Abstract (English)

Large language models (LLMs) with chain-of-thought reasoning have demonstrated remarkable problem-solving capabilities, but controlling their computational effort remains a significant challenge for practical deployment. Recent proprietary systems like OpenAI's gpt-oss series have introduced discrete operational modes for intuitive reasoning control, but the open-source community has largely failed to achieve such capabilities. In this paper, we introduce ThinkDial, the first open-recipe end-to-end framework that successfully implements gpt-oss-style controllable reasoning through discrete operational modes. Our system enables seamless switching between three distinct reasoning regimes: High mode (full reasoning capability), Medium mode (50 percent token reduction with <10 percent performance degradation), and Low mode (75 percent token reduction with <15 percent performance degradation). We achieve this through an end-to-end training paradigm that integrates budget-mode control throughout the entire pipeline: budget-mode supervised fine-tuning that embeds controllable reasoning capabilities directly into the learning process, and two-phase budget-aware reinforcement learning with adaptive reward shaping. Extensive experiments demonstrate that ThinkDial achieves target compression-performance trade-offs with clear response length reductions while maintaining performance thresholds. The framework also exhibits strong generalization capabilities on out-of-distribution tasks.

大模型推理可控生成效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。