通过最短路径剪枝与有限速率门控，大幅减少分布式神经网络通信量。
Sparse In-Network Learning via Shortest-Path Backpropagation and Finite-Rate Gating
- 用最短路径树剪枝非树边，只保留关键通信链路。
- 训练通信量减少70.4%，精度与密集方法相差一个标准差以内。
- 引入有限速率门控，潜变量传输率再降45.7%。
在网学习（INL）通过通信图交换隐层激活和反向传播误差来训练分布式的神经模块。本文提出基于Dijkstra剪枝的INL（D-INL），通过保留以融合节点为根的容量感知最短路径树，移除非树边。为平衡稀疏性与预测信息，本地路由（或聚合）被建模为有限速率随机门控，速率 $R_g=I(Z; T)$。我们推导了率失真-泛化界，并在可复现的分布式分类实验中验证了该方法：D-INL将训练通信量减少70.4%，同时保持精度在密集INL的标准差范围内。加入有限速率正则化后，潜变量估计速率相比未正则化的Dijkstra INL进一步降低45.7%。
原文摘要 · Abstract (English)
In-network learning (INL) trains distributed neural modules by exchanging latent activations and backpropagated errors over a communication graph. This letter proposes Dijkstra-pruned INL (D-INL), which removes non-tree links by retaining a capacity-aware shortest-path tree rooted at the fusion node. To balance sparsity and predictive information, local routing (or aggregation) is modeled as a finite-rate stochastic gate with rate $R_g=I(Z; T)$. We derive a rate-distortion-generalization bound and validate the method on a reproducible distributed-classification experiment, where D-INL reduces training exchange by $70.4\%$ while preserving accuracy within the standard deviation of dense INL. Adding finite-rate regularization further reduces the estimated latent rate by $45.7\%$ relative to unregularized Dijkstra INL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。