arXiv:2608.27265cs.CL2026-08中稿 · EMNLP

揭示大模型推理中缓存数据的因果传递路径,找到真正负责计算的关键组件。

SCIT: Testing Causal Cache Carriers in Latent Chain-of-Thought Models

论文配图:SCIT: Testing Causal Cache Carriers in Latent Chain-of-Thought Models
图 1 · 摘自论文原文
  • 通过构造反事实样本,定位推理过程中关键缓存段的传递机制。
  • 发现算术任务主要依赖值缓存后缀轨迹,而非隐藏状态或键值对整体。
  • 适用于研究模型内部机制、调试大模型推理过程的开发者和研究人员。

隐式思维链模型将中间推理过程隐藏在连续状态中,提升了紧凑性却掩盖了因果对象。本文提出SCIT(后缀缓存交换测试),一种因果协议:构建精确的源-接收反事实,修补声明的缓存段,识别承载反事实计算的Transformer对象。SCIT结合充分性检验与键/值组件拆分、隐状态控制、语义源控制、解码验证及匹配破坏等方法。在CODI-GPT2和模拟CoT风格的GPT-2复现模型上,反事实算术计算主要通过值缓存后缀轨迹传递,而非隐藏状态、键、可重用答案槽或单标记触发。对主版CODI-GPT2检查点,晚值后缀机制具备充分且必要证据;模拟版显示相同充分性与解码控制模式,但匹配破坏证据不足,无法确认必要性。此外,SCIT揭示载体模式转变:类算术的1B GPT-2单元保持隐尾值/KV传递,而8B及修复后的非算术单元则通过提示前缀或完整缓存的K/V路由;边界单元无机制调用。因此,SCIT贡献了缓存级诊断工具、特定检查点的算术机制解析与能力门控载体图谱,而非普适的隐尾主张。

原文摘要 · Abstract (English)

Latent chain-of-thought models move intermediate reasoning from emitted text into continuous states, improving compactness but hiding the causal object. We introduce SCIT, the Suffix Cache Interchange Test, a causal protocol that constructs exact source-recipient counterfactuals, patches declared cache segments, and identifies which transformer object carries the counterfactual computation. SCIT combines sufficiency tests with K/V component splits, hidden-state controls, semantic source controls, decoded validation, and matched corruption. On CODI-GPT2 and a Sim-CoT-style GPT-2 reproduction, counterfactual arithmetic transfers primarily through value-cache suffix trajectories rather than hidden states, keys, reusable answer slots, or single-token triggers. Complete sufficiency-and-necessity evidence for the late-value-suffix mechanism holds for the main CODI-GPT2 checkpoint; the Sim-CoT-style checkpoint shows the same sufficiency and decoded-control pattern but insufficient matched-corruption evidence for a necessity call. Beyond these local arithmetic cells, SCIT reveals carrier-regime shifts: arithmetic-like GPT-2/1B cells preserve latent-tail value/KV transfer, whereas competent 8B and repaired non-arithmetic cells route through prompt-prefix or full-cache K/V; boundary cells receive no mechanism call. SCIT therefore contributes a cache-level diagnostic, a checkpoint-specific GPT-2 arithmetic mechanism, and a competence-gated carrier map rather than a universal latent-tail claim.

因果推理模型解释缓存机制GPT-2

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。