通过分析模型内部状态,精准识别并修复推理失败问题。
SPARK: Susceptibility-Guided Profiling and Steering of Latent Reasoning States in Large Language Models

- 用隐藏状态敏感度诊断模型是否进入有效推理态。
- 在MATH-500上将Qwen3-8B准确率从82.4%提升至85.6%。
- 适合需要提升推理稳定性的大模型部署场景。
大语言模型的推理失败通常仅通过最终答案评估,但错误输出无法揭示根本原因。相同错误可能源于能力缺失、推理轨迹不稳或未激活已有推理状态。现有方法多在输出层操作,通用激活引导则缺乏针对性。本文提出SPARK,通过隐藏状态响应诊断模型是否进入有效推理态,并实现轻量级测试时干预。关键发现:原始隐藏状态敏感度受提示长度干扰,尤其在程序化和算法推理中,更难实例自然更长。为此,SPARK采用长度控制的敏感度,分离输入规模影响与残余推理激活,结合跨层协调选择活跃锚点和激活不足的难题样本。使用FRONTIER-4.5K作为受控程序推理基准进行潜态分析与难度感知评估,在GSM8K和MATH-500上进行前向仅评估。SPARK-Steering持续提升Qwen3系列模型性能;在MATH-500上,Qwen3-4B准确率由82.0%升至84.6%,Qwen3-8B由82.4%升至85.6%。结果表明,敏感度不仅能诊断推理失败,还可作为精准测试时干预的实用指引。
原文摘要 · Abstract (English)
Reasoning failures in large language models (LLMs) are usually evaluated from final answers, but a wrong answer does not reveal why the model failed. The same incorrect output may reflect missing capability, an unstable reasoning trajectory, or a failure to activate a reasoning state that is already available in the frozen model. Existing prompting and benchmark-based evaluation methods mostly operate at the output level, while generic activation-steering methods typically apply global directions without diagnosing which examples require intervention. In this paper, we introduce SPARK, which uses hidden-state response to diagnose whether a model internally enters an effective reasoning state and to guide lightweight test-time steering. The key observation is that raw hidden-state susceptibility is strongly confounded by prompt length, especially in programmatic and algorithmic reasoning where harder serialized instances naturally become longer. SPARK therefore uses length-controlled susceptibility to separate input-scale effects from residual reasoning activation, and combines this signal with cross-layer coordination to select reasoning-active anchors and under-activated hard examples. We use FRONTIER-4.5K as a controlled programmatic reasoning suite for latent profiling and difficulty-aware analysis, and evaluate SPARK-Steering on GSM8K and MATH-500 with forward-only benchmark profiling. Our method improves Qwen3 series models consistently; on MATH-500, accuracy rises from 82.0% to 84.6% for Qwen3-4B and from 82.4% to 85.6% for Qwen3-8B. These results suggest that susceptibility can serve not only as a diagnostic signal for reasoning failures, but also as a practical guide for targeted test-time intervention.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。