提出按物理成本加权的梯度剪枝方法，更省电且不丢性能。
A Theoretical Framework for Energy-Aware Gradient Pruning in Federated Learning
- 根据参数更新的物理成本加权选择重要梯度
- 在非独立同分布CIFAR-10上比Top-K节省更多能量
- 适合资源受限的边缘设备联邦学习场景
联邦学习受分布式边缘设备的通信与能源限制制约。尽管基于Top-K幅度剪枝的梯度稀疏化能有效减少通信负载，但其本质上是能源无关的——它假设所有参数更新的后续传输与内存更新成本相同，忽略了硬件层面的差异。本文将剪枝过程形式化为一个考虑后反向传播阶段内存密集型与计算高效型操作差异的能源约束投影问题。提出代价加权幅度剪枝（CWMP）选择规则，依据参数更新幅度与其物理成本的相对关系进行优先级排序。证明了CWMP是该约束投影问题的最优贪心解，并提供了其全局能源效率的概率分析。在非独立同分布的CIFAR-10基准测试中，数值结果表明CWMP始终在性能-能耗帕累托前沿上优于Top-K基线。
原文摘要 · Abstract (English)
Federated Learning (FL) is constrained by the communication and energy limitations of decentralized edge devices. While gradient sparsification via Top-K magnitude pruning effectively reduces the communication payload, it remains inherently energy-agnostic. It assumes all parameter updates incur identical downstream transmission and memory-update costs, ignoring hardware realities. We formalize the pruning process as an energy-constrained projection problem that accounts for the hardware-level disparities between memory-intensive and compute-efficient operations during the post-backpropagation phase. We propose Cost-Weighted Magnitude Pruning (CWMP), a selection rule that prioritizes parameter updates based on their magnitude relative to their physical cost. We demonstrate that CWMP is the optimal greedy solution to this constrained projection and provide a probabilistic analysis of its global energy efficiency. Numerical results on a non-IID CIFAR-10 benchmark show that CWMP consistently establishes a superior performance-energy Pareto frontier compared to the Top-K baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。