通过步骤标签监控，让语言推理模型更高效生成。
Step-Tagging: Toward controlling the generation of Language Reasoning Models through step monitoring
- 设计轻量级分类器实时标记推理步骤类型。
- 减少20%~50%生成token，准确率基本不变。
- 适合研究模型行为或优化推理效率的人使用。
近年来，语言推理模型（LRMs）在训练与推理技术上持续进步，具备更长、更精准的推理能力。然而，大量研究表明，这些模型仍存在生成冗余验证与反思步骤的问题。为此，本文提出Step-Tagging框架，通过一个轻量级句子分类器实现实时标注模型生成的推理步骤类型。我们构建了新分类体系ReasonType，用于分析推理行为。基于此框架，我们发现在线监控特定步骤数量可形成有效的可解释性提前终止策略。我们在MATH500、GSM8K、AIME、GPQA和MMLU-Pro等标准数据集上评估了三个开源推理模型，结果显示在保持相近准确率的前提下，生成token减少20%至50%，尤其在计算密集型任务中收益显著。本工作为控制语言推理模型生成提供了新方法，也为研究其行为提供了新工具。
原文摘要 · Abstract (English)
The field of Language Reasoning Models (LRMs) has been very active over the past few years with advances in training and inference techniques enabling LRMs to reason longer, and more accurately. However, a growing body of studies show that LRMs are still inefficient, over-generating verification and reflection steps. To address this challenge, we introduce the Step-Tagging framework, a lightweight sentence-classifier enabling real-time annotation of the type of reasoning steps that an LRM is generating. To monitor reasoning behaviors, we introduced ReasonType: a novel taxonomy of reasoning steps. Building on this framework, we demonstrated that online monitoring of the count of specific steps can produce effective interpretable early stopping criteria of LRM inferences. We evaluate the Step-tagging framework on three open-source reasoning models across standard benchmark datasets: MATH500, GSM8K, AIME and non-mathematical tasks (GPQA and MMLU-Pro). We achieve 20 to 50% token reduction while maintaining comparable accuracy to standard generation, with largest gains observed on more computation-heavy tasks. This work offers a novel way to increase control over the generation of LRMs, and a new tool to study behaviors of LRMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。