用张量秩信号优化小模型推理,提升大模型协作效率
RankGuide: Tensor-Rank-Guided Routing and Steering for Efficient Reasoning

- 通过隐藏层张量秩识别小模型推理失败模式
- 路由机制减少1.75倍延迟,准确率媲美大模型
- 适合需要高效推理的数学/代码/科学问答场景
大型推理模型(LRMs)通过生成多步思维链(CoT)提升解题能力,但带来显著推理延迟和计算开销。为缓解此问题,近期研究探索小推理模型(SRMs)与大模型协作的范式,由小模型生成中间推理步骤以实现更好的准确率-延迟权衡。然而,有效且高效地检测和缓解协作系统中SRM的失败仍是关键挑战。本文分析了SRM在生成文本和隐状态空间中的推理过程,识别出三种失效模式:过度自信、不确定性与高重验负担。基于此,提出RankGuide框架,利用连续隐藏状态的张量秩信号作为路由信号,检测SRM可能失败的时刻并选择性调用大模型。此外,引入张量秩过滤的引导向量提取方法,调节小模型的推理轨迹以提升生成质量。通过张量秩信号同时优化路由与引导,RankGuide使协作系统在更少步骤下实现更高效率与准确性。在数学、代码生成、科学问答三个领域实验表明,相比纯大模型,延迟降低最高达1.75倍,且准确率优于或媲美现有方法。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) enhance problem-solving capabilities by generating explicit multi-step chains of thought (CoT) reasoning; however, they incur substantial inference latency and computational overhead. To mitigate this issue, recent works have explored model collaboration paradigms, where small reasoning models (SRMs) generate intermediate reasoning steps to achieve a better accuracy--latency trade-off. Despite recent progress, effectively and efficiently detecting and mitigating SRM failures in collaborative systems remains a key challenge. To address this issue, we analyze SRM inference in both the generated text and hidden-state spaces, and identify three types of failure modes: \textit{overconfidence}, \textit{uncertainty}, and \textit{heavy revalidation}. Building on these insights, we propose \textbf{RankGuide}, a framework that improves the efficiency and effectiveness of SRM--LRM collaboration through tensor-rank-guided routing and steering. Specifically, RankGuide leverages a routing signal that incorporates tensor-rank signals derived from consecutive hidden states to detect when SRMs are likely to fail and selectively invoke LRMs. In addition, we introduce a tensor-rank-filtered steering vector extraction method to modulate the reasoning trajectory of SRMs, thereby improving their generation quality. By improving both routing and steering through tensor-rank signals, RankGuide enables SRM--LRM collaborative systems to achieve more efficient reasoning with fewer steps and improved accuracy. Experiments across three reasoning domains -- mathematics, code generation, and scientific QA -- demonstrate the efficacy of RankGuide in reducing latency by up to $1.75\times$ compared to LRM, while maintaining competitive accuracy relative to prior methods. The code is available at \href{https://github.com/TTTTTTris/RankGuide}{https://github.com/TTTTTTris/RankGuide}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。