发现大模型答选择题分两步：先选内容，再绑定答案符号。
When Models Decide and When They Bind: A Two-Stage Computation for Multiple-Choice Question-Answering
- 通过残差分析与因果干预，揭示模型内部两阶段机制。
- 正确选项的内容信号在最后一项输入后立即可解码。
- 适合研究模型推理机制或提升问答系统准确率的读者。
多项选择题问答（MCQA）虽易于评估，但增加了额外任务：模型不仅需解决问题，还需输出代表答案的符号，导致推理错误与符号绑定失败混淆。本文通过主成分分析（PCA）、线性探测和因果干预，研究语言模型内部实现MCQA的方式。结果发现，选项分隔符（换行）处的残差状态常包含与各选项正确性相关的强线性可解码信号。赢家身份探测显示，两个阶段的进展：获胜内容位置在最后一选项处理后立即可解码，而输出符号则在答案生成位置附近才被表示。在符号与内容置换测试中，均支持模型先在内容空间选择胜者，再将该胜者绑定到对应输出符号的两阶段机制。
原文摘要 · Abstract (English)
Multiple-choice question answering (MCQA) is easy to evaluate but adds a meta-task: models must both solve the problem and output the symbol that *represents* the answer, conflating reasoning errors with symbol-binding failures. We study how language models implement MCQA internally using representational analyses (PCA, linear probes) as well as causal interventions. We find that option-boundary (newline) residual states often contain strong linearly decodable signals related to per-option correctness. Winner-identity probing reveals a two-stage progression: the winning *content position* becomes decodable immediately after the final option is processed, while the *output symbol* is represented closer to the answer emission position. Tests under symbol and content permutations support a two-stage mechanism in which models first select a winner in content space and then bind or route that winner to the appropriate symbol to emit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。