通过分阶段优势估计提升大模型推理效率，减少冗余输出
SHAPE: Stage-aware Hierarchical Advantage via Potential Estimation for LLM Reasoning
- 将推理过程建模为可解性状态空间的轨迹，分段评估进展
- 在三个基础模型上实现平均准确率提升3%，令牌消耗减少30%
- 适合需要高效推理的数学题求解场景，尤其关注少而精的生成
过程监督已成为提升大模型推理能力的有前途方法，但现有方法难以区分有意义进展与单纯冗长表达，导致推理能力受限且令牌使用效率低下。为此，我们提出分阶段层次优势估计框架（SHAPE），将推理过程形式化为可解性状态空间中的轨迹。该框架引入分层信用分配机制：在片段层面，采用分阶段优势函数，优先推动低潜力状态下的高效突破；在令牌层面，利用熵驱动重分配以强化执行信号。在三种基础模型和五个基准上的数学推理实验表明，SHAPE实现了平均准确率提升3%，同时令牌消耗降低30%。
原文摘要 · Abstract (English)
Process supervision has emerged as a promising approach for enhancing LLM reasoning, yet existing methods fail to distinguish meaningful progress from mere verbosity, leading to limited reasoning capabilities and unresolved token inefficiency. To address this, we propose Stage-aware Hierarchical Advantage via Potential Estimation (SHAPE), a framework that formalizes reasoning as a trajectory through a state space of empirical solvability. SHAPE introduces a hierarchical credit assignment mechanism: at the segment level, it employs a stage-aware advantage function to prioritize efficient breakthroughs in low-potential states; at the token level, it utilizes entropy-driven redistribution to sharpen execution signals. Extensive experiments in math reasoning across three base models and five benchmarks demonstrate that SHAPE achieves an average accuracy gain of 3% with 30% reduced token consumption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。