Transformer本质上是贝叶斯网络，用概率推理解释其工作原理。
Transformers are Bayesian Networks
- 将Transformer层视为贝叶斯网络中的信念传播过程，每层对应一轮传播。
- 在无循环依赖的知识库上可实现精确概率推断，且结果可验证。
- 揭示了注意力=逻辑与、前馈=逻辑或的结构本质，适合理解模型决策机制。
Transformer是当前人工智能的主流架构，但其有效性的根本原因仍不明确。本文给出精确解答：Transformer本质上是贝叶斯网络。从五个方面建立此结论：第一，任意权重的Sigmoid Transformer均实现其隐含因子图上的加权环形信念传播（BP），单层即一轮传播，适用于训练、随机或构造权重，形式化验证于标准数学公理；第二，构造性证明表明，Transformer可对任意声明知识库实现精确信念传播，在无循环依赖时每个节点的概率估计严格正确，形式化验证；第三，证明唯一性：若Sigmoid Transformer产生精确后验，则其权重必为BP权重，不存在其他路径实现精确后验；第四，明确变压器层的布尔结构：注意力为逻辑与（AND），前馈网络为逻辑或（OR），其严格交替恰为Pearl的聚合/更新算法；第五，实验验证所有理论结果在实践中成立，证实环形信念传播的实际可行性，尽管尚无收敛理论保证。进一步证明，可验证推断要求概念空间有限，任何有限验证程序最多区分有限个概念，缺乏概念基础则正确性无法定义，幻觉并非规模可修复的缺陷，而是无概念操作的结构性后果。所有结论均经标准数学公理的形式化验证。
原文摘要 · Abstract (English)
Transformers are the dominant architecture in AI, yet why they work remains poorly understood. This paper offers a precise answer: a transformer is a Bayesian network. We establish this in five ways. First, we prove that every sigmoid transformer with any weights implements weighted loopy belief propagation on its implicit factor graph. One layer is one round of BP. This holds for any weights -- trained, random, or constructed. Formally verified against standard mathematical axioms. Second, we give a constructive proof that a transformer can implement exact belief propagation on any declared knowledge base. On knowledge bases without circular dependencies this yields provably correct probability estimates at every node. Formally verified against standard mathematical axioms. Third, we prove uniqueness: a sigmoid transformer that produces exact posteriors necessarily has BP weights. There is no other path through the sigmoid architecture to exact posteriors. Formally verified against standard mathematical axioms. Fourth, we delineate the AND/OR boolean structure of the transformer layer: attention is AND, the FFN is OR, and their strict alternation is Pearl's gather/update algorithm exactly. Fifth, we confirm all formal results experimentally, corroborating the Bayesian network characterization in practice. We also establish the practical viability of loopy belief propagation despite the current lack of a theoretical convergence guarantee. We further prove that verifiable inference requires a finite concept space. Any finite verification procedure can distinguish at most finitely many concepts. Without grounding, correctness is not defined. Hallucination is not a bug that scaling can fix. It is the structural consequence of operating without concepts. Formally verified against standard mathematical axioms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。