动态调节推理熵，让大模型简单问题简洁回答，难题则深入思考。
Taming the Thinker: Conditional Entropy Shaping for Adaptive LLM Reasoning
- 基于分枝点熵值动态调控，区分简单与复杂任务的输出策略。
- 在12个数学基准上平均准确率提升，响应长度比基线减少17%以上。
- 适用于需要平衡精度与效率的场景，尤其适合教学与交互式推理。
基于熵的深度推理已成为提升大语言模型（LLMs）推理能力的有前景方向，但现有方法或盲目增加响应长度，或为缩短长度牺牲准确性。为此，我们提出条件熵调控（CES）框架，通过动态控制逐标记响应熵，使模型在简单问题上生成简洁答案，在复杂问题上鼓励深层探索。该框架基于DAPO，将逐标记熵作为不确定性信号，采用条件双向策略：在正确路径上惩罚高熵“分叉点”标记以提升简洁性；在错误路径上奖励高熵标记以促进探索和纠错。我们在DeepSeek-R1-Distill-7B上实现CES，评估其在12个数学基准上的表现。结果表明，相较于DAPO，CES在保持甚至提升平均准确率的同时，显著降低响应长度；补充实验显示，该趋势在1.5B规模的骨干模型及跨域基准上同样成立。
原文摘要 · Abstract (English)
Entropy-based deep reasoning has emerged as a promising direction for improving the reasoning capabilities of Large Language Models (LLMs), but existing methods often either increase response length indiscriminately or shorten responses at the cost of accuracy. To better balance this trade-off, we introduce Conditional Entropy Shaping (CES), a framework that dynamically controls token-level response entropy, enabling LLMs to produce concise solutions on simple problems while encouraging deeper exploration on hard ones. Built on DAPO, CES uses token-level entropy as an uncertainty signal and applies a conditional bidirectional policy: it penalizes high-entropy "forking point" tokens on correct reasoning paths to improve conciseness, and rewards them on incorrect paths to encourage exploration and error correction. We implement CES on DeepSeek-R1-Distill-7B and evaluate it on 12 mathematical benchmarks. CES consistently improves average accuracy while reducing response length relative to DAPO, and supplementary experiments show similar trends on a smaller 1.5B backbone and on out-of-domain benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。