提出局部计算雅可比转置的新方法，让神经网络误差传播无需全局反向传播。
Weight Feedback Computes the Jacobian Transpose Locally in Modern Deep Networks
- 利用激活与归一化层的局部信息，直接计算雅可比转置的分解形式。
- 在CIFAR-10上深度增加时准确率持续提升，比经典PC方法高2.7至22.3个百分点。
- 适用于追求生物可解释性与低通信开销的深度网络训练，适合神经形态计算场景。
预测编码（PC）通过局部权重更新提供一种生物启发的替代反向传播的方法，但层间误差传递仍依赖自动微分的雅可比转置（$J^ op$）——这是PC中唯一的非局部操作。我们证明这种依赖在多数现代深层网络中可被消除。对于任意形如 $f(x)=\mathrm{Act}(\mathrm{Norm}(L(x)))$ 且归一化统计量固定的层，精确的 $J^ op$ 可分解为三个局部可得项：$J^ op v = L^ op(s \odot σ'(z) \odot v)$，其中 $σ'$ 为激活函数导数，$z$ 为预激活值，$s=γ/σ_{\mathrm{run}}$ 为归一化增益。此前的权重反馈方法忽略了这两项修正；恢复后可完全关闭该层的误差传输瓶颈。局部性成立基于三项假设：权重对称性（$L^\top$ 对应前向算子，所有PC均假设如此）、非突触局部的软谱范数控制，以及最大池化最近邻近似。将该恒等式代入PC即得WF-Act-PC，彻底移除误差传输中的反向传播。在CIFAR-10/100上（50轮，5次随机种子），只有此方法随深度增加而性能提升，在CIFAR-10上超越iPC基线2.7–22.3个百分点。调优后，其在更深的CIFAR-10架构（VGG-9: 93.57% vs. 92.43%；ResNet-18: 92.76% vs. 91.54%）和更难的Tiny-ImageNet上达到或超过同规模反向传播基线，仅在更深的CIFAR-100 VGG模型上略低于调优后的反向传播。
原文摘要 · Abstract (English)
Predictive Coding (PC) offers a biologically motivated alternative to backpropagation via local weight updates, yet routing error between layers still relies on an autograd Jacobian-transpose ($J^\top$) product - the last non-local operation in PC. We show that this dependency is largely avoidable. For any layer $f(x)=\mathrm{Act}(\mathrm{Norm}(L(x)))$ with frozen normalization statistics, the exact $J^\top$ factors into three locally available terms, $J^\top v = L^\top(s \odot σ'(z) \odot v)$, where $σ'$ is the activation derivative, $z$ is the pre-activation, and $s=γ/σ_{\mathrm{run}}$ is the normalization gain. Prior weight-feedback methods omitted both corrections; restoring them closes the transport gap for this layer class. Locality here holds up to three assumptions, which we state upfront: weight symmetry ($L^\top$ mirrors the forward operator, as assumed by all PC), a soft spectral-norm control that is not synapse-local, and a nearest-neighbour approximation for MaxPool. Substituting the identity into PC yields WF-Act-PC, which removes the autograd backward pass from error transport. On CIFAR-10/100 (50 epochs, 5 seeds), WF-Act-PC is the only PC method whose accuracy improves with depth, surpassing iPC - the strongest classical PC baseline - by 2.7-22.3 pp on CIFAR-10. With both methods tuned per architecture, it matches or exceeds a comparably-tuned backpropagation baseline on the deeper CIFAR-10 architectures (VGG-9: 93.57% vs. 92.43%; ResNet-18: 92.76% vs. 91.54%) and on the harder Tiny-ImageNet benchmark, while trailing tuned BP on the deeper CIFAR-100 VGG cells. Our WF-Act-PC implementation is publicly available at https://github.com/jlshen025/pcax
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。