神经棋手会算对棋路却故意不走，因安全偏好压倒了正确计算。
The Algorithm Is Not the Behavior: Learned Priors Override Look-Ahead in a Chess-Playing Neural Network
- 发现模型内部能正确推演棋局，但最终输出被安全优先策略覆盖
- 61.7%的错误答案可通过强制偏离安全偏好恢复正确解
- 揭示算法结构不等于行为表现，安全偏好可压制内部计算
近期机制研究在神经网络中发现了模块化算术、游戏中的搜索与规划等学习到的算法。但算法结构是否保证算法行为？我们以最强神经棋手Leela Chess Zero为例，其先前研究已识别出学习到的看棋能力。通过将逻辑透镜扩展至其走子策略网络，我们发现正确解（包括直接将死）常出现在中间层，但在最终输出中被系统性覆盖，此现象称为“遗忘谜题”。对这些局面复现前期分析，发现看棋功能正常——正确后续步法被表征，具有因果重要性且可线性解码，排除了算法失效。相反，后期层逐渐偏向安全下法而非攻击。通过引导模型违背此偏好，成功恢复61.7%的遗忘谜题，提供因果证据：安全先验确实覆盖了算法计算结果。这表明，即使内部正确求解，模型仍可能输出错误答案。
原文摘要 · Abstract (English)
Recent mechanistic work has uncovered learned algorithms within neural networks, from modular arithmetic to search and planning in game-playing agents. But does algorithmic structure guarantee algorithmic behavior? We investigate this in Leela Chess Zero, the strongest neural chess engine, where prior work identified learned look-ahead. By extending the logit lens to its move-selecting policy network, we discover that correct puzzle solutions-including immediate checkmates-often appear in intermediate layers but are systematically overridden in the final output, a phenomenon we term "forgotten puzzles". Replicating prior analyses on these positions, we find that look-ahead operates normally-future moves of the correct continuation are represented, causally important, and linearly decodable-ruling out a failure of the algorithm itself. Instead, late layers increasingly shift toward prioritizing safe play over aggression. To test whether this shift drives the override, we steer the model against these preferences and recover 61.7% of forgotten puzzles, providing causal evidence that safety priors override algorithmically computed solutions. These findings demonstrate that algorithmic structure does not guarantee algorithmic behavior: a model can internally solve a problem and still output the wrong answer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。