用图结构分析推理链，自动识别大模型的冗余步骤
CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs
- 将自由文本推理链转为有向依赖图，提取最短有效路径
- 21个大模型评估显示普遍存在冗余，平均冗余率未量化但普遍存在
- 适合关注模型推理效率与可解释性的研究者
大型推理模型（LRMs）通过生成长篇推理链（CoT）来提升表现，但常伴随过度推理：重复计算和循环自验证，增加计算开销却不改善结果。现有评估多关注最终准确率或粗略的词元数量，缺乏自动化工具区分必要逻辑与结构性冗余。本文提出 CoTJudger，一种基于图的框架，将自由形式的 CoT 转换为有向依赖图，并提取达到正确答案所需的最短有效路径（SEP），从而量化推理效率。该方法提供可解释的效率信号——揭示多少内容是必要的，多少属于结构冗余，且可在不同模型和任务间比较。对 21 个 LRMs 的评估发现冗余普遍存在，并暴露了反复出现的失败模式，如‘验证执念’和‘补偿性冗余’。该结果为分离推理能力与计算浪费提供了实用度量，支持更精准的模型评估与诊断。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) have demonstrated strong performance by producing extended Chain-of-Thought (CoT) traces before answering. However, this paradigm often induces over-reasoning: redundant calculations and circular self-verification that increase computational cost without improving outcomes. Existing evaluations largely emphasize final accuracy or coarse token counts, and lack automated tools to separate essential logic from structural redundancy. We introduce CoTJudger, a graph-driven framework that quantifies reasoning efficiency by converting free-form CoTs into directed dependency graphs and extracting the Shortest Effective Path (SEP) needed to reach a correct solution. This yields an interpretable efficiency signal -- how much of a CoT is necessary versus structurally redundant -- that is comparable across models and tasks. Evaluating 21 LRMs, CoTJudger reveals pervasive redundancy and surfaces recurring failure modes, including verification obsession and compensatory redundancy. These results provide a practical metric for disentangling reasoning ability from computational waste, enabling more targeted evaluation and diagnosis of LRM efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。