通过一次干预修改缓存,让大模型自动展开推理过程。
KV Cache Steering for Controlling Frozen LLMs
- 直接操作键值缓存,用一句话改变模型推理方式。
- 在GPQA和MATH等难题上提升性能,小模型效果显著。
- 无需微调或改提示,适合快速集成到现有系统中。
我们提出缓存引导(cache steering),一种轻量级的隐式控制语言模型的方法,通过一次干预直接作用于键值缓存。为验证有效性,我们将该方法应用于诱导小型语言模型产生链式思维推理。通过从教师模型(如GPT-4o)或人工标注中获取推理轨迹,构建引导向量,使模型行为转向更明确、多步骤的推理,且无需微调或修改提示。在多个推理基准上的实验表明,缓存引导不仅提升了推理过程的结构质量,还显著提高了任务表现。额外实验显示,该方法可扩展至大型模型,并在挑战性数据集如GPQA和MATH上带来进一步提升。相比以往需要持续干预的激活引导技术,本方法在推理延迟、超参数稳定性及与现有推理API的集成便捷性方面具有显著优势。此外,缓存引导还能实现推理风格(如分步式、因果式、类比式)的可控迁移,成为语言模型行为层面引导的实用工具。
原文摘要 · Abstract (English)
We propose cache steering, a lightweight method for implicit steering of language models via a one-shot intervention applied directly to the key-value cache. To validate its effectiveness, we apply cache steering to induce chain-of-thought reasoning in small language models. Our approach constructs steering vectors from reasoning traces, obtained either from teacher models (e.g., GPT-4o) or existing human annotations, that shift model behavior toward more explicit, multi-step reasoning without fine-tuning or prompt modifications. Experimental evaluations on diverse reasoning benchmarks demonstrate that cache steering improves both the qualitative structure of model reasoning and quantitative task performance. Additional experiments show that the method also scales to larger models and yields further gains on challenging datasets such as GPQA and MATH. Compared to prior activation steering techniques that require continuous interventions, our one-shot cache steering offers substantial advantages in terms of inference latency, hyperparameter stability, and ease of integration with existing inference APIs. Beyond mere reasoning induction, we show that cache steering enables controllable transfer of reasoning styles (e.g., stepwise, causal, analogical), making it a practical tool for behavior-level guidance of language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。