arXiv:2607.27617cs.AI2026-07被引 2

发现语言模型中可复用的因果接口,提升未来计算的可预测性。

Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures

论文配图:Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures
图 1 · 摘自论文原文
  • 通过分支未来采样比较隐藏状态,构建无需人工标注的因果接口。
  • 共享接口在两个模型上分别降低0.216和0.294 nats描述长度。
  • 适合研究模型内部机制、可解释性与接口复用的学者。

相同语言模型输出可能源于支持不同未来计算的隐藏状态,因此仅依赖当前输出的探测无法建立可复用的内部接口。本文提出“分支未来”:在前缀状态形成后才采样未来操作,并通过这些操作诱导的响应分布来比较状态。该方法在不依赖研究者指定潜在标签的情况下,得出隐藏状态的实证因果商。随后,共享、局部、混合与分布式接口在预序因果描述长度下竞争,需满足未来签名保真度与容量约束。在两个模型评估中,共享接口保持最低的保留描述长度,分别在Qwen2.5-1.5B和Llama-3-8B上获得0.216和0.294 nats的增益,且均维持紧密聚类的平均未来签名失真;五骨干扫描验证了共享优势方向一致。图对齐移植分析显示,共享接口在目标正确性、局部性、复制保持与综合表现上最强,其路径中介效应达0.749,远高于匹配零路径的0.150。盲测四分类模型生物测试中,16个架构中有14个被正确恢复,非共享到共享的误判仅1例(共12个非共享对象)。结果表明,在测试的操作库范围内存在经济高效的可复用因果接口,但结论明确限于候选架构、干预方式及保留未来场景。

原文摘要 · Abstract (English)

Identical language-model answers can arise from hidden states that support different future computations, so current-answer probes do not establish a reusable internal interface. We introduce forked futures: future operations are sampled only after a prefix state has formed, and states are compared through the response distributions induced by those operations. This yields an empirical causal quotient over hidden states without requiring researcher-specified latent labels. Shared, Local, Mixture, and Distributed interfaces then compete under prequential causal description length subject to future-signature fidelity and matched capacity constraints. In the two detailed model evaluations, Shared has the lowest held-out description length, with gains of 0.216 nats on Qwen2.5-1.5B and 0.294 nats on Llama-3-8B, while maintaining tightly clustered mean future-signature distortion; a five-backbone sweep preserves the positive direction of Sharedness Gain. The figure-aligned transplantation analysis gives Shared the strongest joint target-correctness, locality, copy-preservation, and composite profile, and API-aligned paths mediate 0.749 of the target effect versus 0.150 for matched null paths. In the blind four-class model-organism test, 14/16 architectures are recovered, with one observed non-Shared to Shared error among 12 non-Shared organisms. These results support an economical reusable causal interface within the tested operation banks, while keeping the claim explicitly conditional on the candidate architectures, interventions, and held-out futures.

语言模型因果接口隐藏状态可复用性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。