arXiv:2504.07986cs.CLcs.AI2025-04被引 75

通过可调控的推理校准,让大模型少走弯路,又快又准。

SEAL: Steerable Reasoning Calibration of Large Language Models for Free

  • 分析推理路径,将思考分为执行、反思、过渡三类,发现冗余来自过度反思与过渡
  • 无需训练,用潜空间向量干预推理,准确率最高提升11%,推理token减少超一半
  • 方法通用性强,跨模型跨任务都能用,适合想提速提准的开发者

大语言模型(如OpenAI的o1系列)通过扩展链式思维(CoT)机制在复杂推理任务中表现出色。然而,近期研究发现CoT推理轨迹存在大量冗余,不仅增加推理延迟,还因分散注意力而降低性能。我们分析了模型内部推理结构,将其分为执行、反思和过渡三类思维,并发现过度反思与过渡思维与失败案例强相关,且三类思维在潜空间中具有明显分离特征。基于此,提出SEAL(可调控推理校准)——一种无需训练的方法,通过离线提取潜空间中的推理引导向量,再在线动态干预推理过程以优化结果。该方法显著提升准确率,同时大幅减少推理开销:在DeepSeek-R1-Distill和QwQ-32B-Preview等模型上,于Math500、GSM8K、LiveCodeBench等多个基准测试中,准确率最高提升11%,推理token减少11.8%至50.4%。代码已开源。

原文摘要 · Abstract (English)

Large Language Models (LLMs), such as OpenAI's o1-series have demonstrated compelling capabilities for complex reasoning tasks via the extended chain-of-thought (CoT) reasoning mechanism. However, recent studies reveal substantial redundancy in the CoT reasoning traces, which not only increases inference latency but also negatively impacts model performance by diverting attention to unnecessary reasoning paths. To address this issue, we investigate the internal reasoning structures of LLMs and categorize them into three primary thought types: execution, reflection, and transition thoughts. Moreover, our analysis reveals that excessive reflection and transition thoughts are strongly correlated with failure cases and these thought categories exhibit clear separation in the latent space. Based on these, we introduce SEAL (Steerable reasoning calibration), a training-free approach that seamlessly calibrates the CoT process, improving accuracy while demonstrating significant efficiency gains. SEAL consists of an offline stage for extracting the reasoning steering vector in the latent space, followed by an on-the-fly calibration of the reasoning trace through representation intervention using the steering vector. Notably, the steering vector exhibits strong transferability across various tasks. Extensive experiments across multiple models (DeepSeek-R1-Distill and QwQ-32B-Preview) and benchmarks (Math500, GSM8K, LiveCodeBench) validate the effectiveness of SEAL, up to a 11% improvement in accuracy while reducing reasoning tokens by 11.8% to 50.4%. Our code is publicly available at https://github.com/VITA-Group/SEAL.

大模型推理链式思维效率优化无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。