分析大模型如何理解因果与对立关系，揭示其内部推理机制
For What Reason? Interpreting Models' Encoding of Causation and Antithesis
- 将对话关系建模为下一个词预测任务，用可解释性方法探测模型内部
- 早期层在序列中段做出判断，中层接近结尾才最终决定
- 部分层对特定答案有偏好，体现推理表征的不对称性
话语关系为文档提供结构，对语言理解至关重要，直接影响语言模型的表现与伦理性。本文研究指令微调的Transformer模型（LLaMA和Mistral）在英文中对话语关系的编码方式，重点关注因果与对立关系。将任务设为下一个词预测，并运用一系列可解释性技术分析模型内部机制。结果表明，某些早期层在序列中段即做出预测性判断，而部分中层则在接近末尾时才完成决策。其余大部分层主要传递前期判断，而非主动影响决策。此外，观察到某些层对某一答案存在偏好，暗示基于话语的推理存在非对称表征。
原文摘要 · Abstract (English)
Discourse relations provide document structure, critical to language understanding and enabling language model performance and ethicality. In this work, we investigate how instruction-tuned Transformer models (LLaMA and Mistral) encode discourse relations in English, with a particular focus on the contrasting relations of causation and antithesis. Framing the task as a next-token prediction task and applying a suite of interpretability techniques to test model internals, our findings show that certain early layers make predictive decisions at mid-sequence tokens, while some mid-level layers finalize their decisions closer to the last token. Most of the remaining layers primarily propagate earlier decisions rather than actively influencing them. Additionally, we observe that some layers exhibit a preference for one answer over alternatives, suggesting asymmetric representation of discourse-based reasoning.\footnote{Our code is available at https://github.com/abhidipbhattacharyya/causation_vs_antithesis}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。