通过标记推理步骤实现动态早停,显著减少语言模型的计算量。
TRACES: Tagging Reasoning Steps for Adaptive Cost-Efficient Early-Stopping

- 实时标记推理步骤类型,根据行为变化判断是否提前终止
- 在多个数学与知识推理任务上实现20%至50%的令牌节省
- 适用于希望提升推理效率的研究者与部署团队
近年来,语言推理模型(LRMs)在训练和推理技术上取得进展,能进行更长且更准确的推理。然而,研究显示这些模型仍存在生成冗余验证与反思步骤的问题,且各推理步骤的高层次作用及其对正确答案的贡献尚不明确。为此,我们提出TRACES(Tagging of the Reasoning steps enabling Adaptive Cost-Efficient early-Stopping),一个轻量级框架,可在推理过程中实时标注推理步骤,并实现自适应、成本高效的早停。通过监测推理行为,我们发现模型在得出正确答案后会改变推理模式。结果显示,监控特定步骤类型可形成有效的可解释早停标准。在三个数学推理基准(MATH500、GSM8K、AIME)和两个知识与推理基准(MMLU、GPQA)上,TRACES实现了20%至50%的令牌减少,同时保持与标准生成相当的准确率。在更难的任务(BeyondAIME、IMO AnswerBench)上,精度-效率权衡更陡峭,需采用更保守的早停阈值。该工作为研究和控制LRMs的生成行为提供了新途径。
原文摘要 · Abstract (English)
The field of Language Reasoning Models (LRMs) has been very active over the past few years with advances in training and inference techniques enabling LRMs to reason longer, and more accurately. However, a growing body of studies show that LRMs are still inefficient, over-generating verification and reflection steps. Additionally, the high-level role of each reasoning step and how different step types contribute to the generation of correct answers, is largely underexplored. To address this challenge, we introduce TRACES (Tagging of the Reasoning steps enabling Adaptive Cost-Efficient early-Stopping), a lightweight framework that tags reasoning steps in real-time, and enable adaptive, cost-efficient early stopping of large-language-model inferences. By monitoring reasoning behaviors during inferences, we find that LRMs tend to shift their reasoning behavior after reaching a correct answer. We demonstrate that the monitoring of the specific type of steps can produce effective interpretable early stopping criteria. Evaluated on three mathematical reasoning benchmarks (MATH500, GSM8K, AIME) and two knowledge and reasoning benchmarks (MMLU and GPQA), TRACES achieve 20 to 50% token reduction while maintaining comparable accuracy to standard generation. On harder tasks (BeyondAIME, IMO AnswerBench), the accuracy-efficiency trade-off is steeper, requiring a more conservative early-stopping threshold. This work offers a novel way to study and control the generation behavior of LRMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。