让大模型少废话，用熵值判断冗余，又快又准
ENTRA: Entropy-Based Redundancy Avoidance in Large Language Model Reasoning
- 基于双向重要性估计，量化每个词的推理价值
- 通过低重要性词的熵值惩罚，使输出长度减少37%~53%
- 适合想提升推理效率又不牺牲准确率的研究者
大语言模型在推理时常陷入过度思考，生成冗长链路却无实质提升。本文提出ENTRA，一种基于熵的训练框架，有效抑制冗余推理。ENTRA首先采用轻量级双向重要性估计（BIE）方法，综合预测置信度与前向影响评估词级重要性；随后计算低重要性词的熵值奖励，按理论上限归一化后，通过强化学习优化。数学推理基准测试显示，ENTRA在不损失甚至提升准确率的前提下，将输出长度减少37%至53%。该方法为减少大模型过思现象提供了系统性且可泛化的解决方案。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) often suffer from overthinking, generating unnecessarily long reasoning chains even for simple tasks. This leads to substantial computational overhead with limited performance gain, primarily due to redundant verification and repetitive generation. While prior work typically constrains output length or optimizes correctness, such coarse supervision fails to guide models toward concise yet accurate inference. In this paper, we propose ENTRA, an entropy-based training framework that suppresses redundant reasoning while preserving performance. ENTRA first estimates the token-level importance using a lightweight Bidirectional Importance Estimation (BIE) method, which accounts for both prediction confidence and forward influence. It then computes a redundancy reward based on the entropy of low-importance tokens, normalized by its theoretical upper bound, and optimizes this reward via reinforcement learning. Experiments on mathematical reasoning benchmarks demonstrate that ENTRA reduces output length by 37% to 53% with no loss-and in some cases, gains-in accuracy. Our approach offers a principled and efficient solution to reduce overthinking in LRMs, and provides a generalizable path toward redundancy-aware reasoning optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。