尝试用激活向量传递推理状态，结果失败。
A Negative Result on Cross-Model Activation Transfer in a Pythia Multi-Hop Setting
- 用线性映射对齐两模型中间层激活态
- 跨模型激活传递未能提升推理准确率
- 适合研究模型间通信机制的边界
近期研究表明，语言模型可通过生成数据中的隐藏信号在训练中传递行为特征。本文探讨另一种激活驱动的通信方式：能否在推理阶段通过后处理线性激活桥，将一个模型的中间推理状态传递给另一个模型？我们在 Pythia-160M 到 Pythia-410M 的多跳推理任务中测试该设想。线性映射层在不同随机种子下实现了接近 0.97 的归一化余弦相似度，表明激活空间对齐良好。然而，当将转换后的激活注入接收模型时，下游任务表现未提升。弱强度加法注入与无注入基线无显著差异（置信区间包含零），替换式注入则始终造成性能下降。将译向量按接收模型激活范数缩放也无法挽救效果。结论为：在此设定下，离线表征对齐不足以实现接收模型内的有效因果通信。
原文摘要 · Abstract (English)
Recent work shows that language models can transmit behavioural traits through hidden signals in generated data during training. We ask whether a different activation-mediated channel is viable: can one language model communicate a useful intermediate reasoning state to another at inference time through a post-hoc linear activation bridge, rather than through a textual or structured-token relay? We test this question in a controlled Pythia-160M to Pythia-410M multi-hop reasoning setting. A linear translation layer learns a strong normalized-space map between sender and receiver hidden states, with normalized cosine similarity near 0.97 across seeds. However, when the translated activations are injected into the receiver at inference time, they do not improve downstream answering. Low-strength additive injection remains near the no-injection baseline, with confidence intervals that cross zero. Replacement-style injection is consistently destructive, and rescaling translated vectors to the receiver hidden-state norm does not rescue performance. The result is therefore a scoped negative result: in this setting, offline representational alignment is not sufficient for useful causal communication inside the receiver.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。