用层叠上同调理论分析线性预测编码网络的误差根源
Sheaf Cohomology of Linear Predictive Coding Networks
- 将预测编码网络建模为细胞层叠，激活值通过上边界映射生成边级预测误差
- 发现反馈环路引发内在矛盾误差，导致学习停滞，且可用霍奇分解识别
- 提供诊断工具与权值初始化设计原则，适合研究神经网络优化机制者
预测编码（PC）用局部优化替代全局反向传播。我们证明线性PC网络可自然表述为细胞层叠：层叠上边界映射将激活值转为边级预测误差，而PC推断等价于在层叠拉普拉斯算子下的扩散过程。层叠上同调刻画了推理无法消除的不可约误差模式。我们分析具有反馈环路的递归拓扑，发现其产生与监督无关的预测误差。通过霍奇分解，确定这些矛盾何时导致学习停滞。该层叠形式化既提供诊断工具以识别问题网络结构，也给出递归PC网络的有效权值初始化设计原则。
原文摘要 · Abstract (English)
Predictive coding (PC) replaces global backpropagation with local optimization over weights and activations. We show that linear PC networks admit a natural formulation as cellular sheaves: the sheaf coboundary maps activations to edge-wise prediction errors, and PC inference is diffusion under the sheaf Laplacian. Sheaf cohomology then characterizes irreducible error patterns that inference cannot remove. We analyze recurrent topologies where feedback loops create internal contradictions, introducing prediction errors unrelated to supervision. Using a Hodge decomposition, we determine when these contradictions cause learning to stall. The sheaf formalism provides both diagnostic tools for identifying problematic network configurations and design principles for effective weight initialization for recurrent PC networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。