发现大模型不仅知道结论对错，还记住推理过程的语义结构。
Is Inference Mediated by Distinct Semantic Structures in LLMs? A Mechanistic Interpretation
- 用奇异值分解提取各层中语义操作的独立子空间。
- 语义变换可准确解码，正确率达84.8%至99%。
- 不同模型间操控效果差异大，揭示推理机制复杂性。
预测标签正确并不一定需要表示生成该标签的操作。虽然已知Transformer的表征包含标签级信息，但其是否编码产生这些标签的语义操作尚不明确。本文在自然语言推理任务中，使用仅差一个语义转换的受控前提-假设对进行研究。通过分析层间激活，利用奇异值分解（SVD）估计操作级子空间，并在四个开源解码器模型中通过激活操纵测试其因果相关性。结果表明，语义变换可实现84.8%–99%的解码准确率，且占据部分独立但重叠的子空间，显著优于随机子空间基线。操纵实验显示这些方向对预测有因果影响，尽管不同模型间操控能力存在差异；跨操作操纵进一步揭示了结构化干扰现象，并发现子空间选择性与跨操作独立性之间存在分离。这说明模型不仅编码了假设与前提的关系，还在一定程度上编码了这种关系的生成方式，提示机制解析与控制应聚焦于语义操作层面，而非仅限于预测标签。
原文摘要 · Abstract (English)
Predicting a label correctly does not necessarily require representing the operation that produces it. Transformer representations are known to carry label-level information, but whether they encode semantic operations producing those labels is unclear. We investigate this in Natural Language Inference using controlled premise-hypothesis pairs that differ by a single semantic transformation. Using layer-wise activations, we estimate operation-level subspaces via SVD and test their causal relevance through activation steering in four open-weight decoder models. Transformation effects are decodable with $84.8$-$99\%$ accuracy and occupy partially distinct but overlapping subspaces, exceeding random-subspace baselines. Steering experiments show that these directions causally influence predictions, though steerability varies across models; cross-operation steering further reveals structured interference and a dissociation between subspace selectivity and cross-operation independence. These findings indicate that the models encode not only that a hypothesis relates to a premise but also, in part, how it does so, implying that mechanistic analysis and control should operate at the level of semantic operations rather than predicted labels alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。