FedRAN用压缩统计量替代梯度,实现高效低资源持续联邦学习。
Accurate and Resource-Efficient Federated Continual Learning

- 用截断SVD压缩梯度矩阵,通信量从二次方降为线性
- 在多个数据集上准确率提升4.8个百分点,通信量减少30倍以上
- 支持标签稀缺场景,仅20%标签即可提升6.61点准确率
联邦持续学习(FCL)需在通信、计算、内存和标签有限的条件下,从分布式任务流中学习。现有方法常依赖重复本地优化、回放和全监督,而解析方法虽避免迭代训练与回放,但使用高维随机特征提升精度需二阶统计量——核矩阵,其通信成本随随机特征数M呈二次增长。本文提出FedRAN,一种资源感知的解析式FCL框架,以紧凑随机特征统计量替代梯度更新。客户端传输其核矩阵的截断SVD摘要,将主导的二阶上传成本从关于M的二次降至线性(固定秩下)。服务器执行两层QR-SVD子空间融合:跨客户端的空间合并与跨任务的时间合并,并闭式求解岭分类器。此外,通过基于原型的伪标签机制支持标签稀缺场景。在CIFAR-100、ImageNet-R和VTAB数据集上,FedRAN相比最强基线平均准确率提升最高达4.8个百分点,单客户端通信量比优化型方法降低30.6–121.8倍,平均速度提升190.3倍;仅使用20%标签时,伪标签使平均准确率提升最高达6.61点。结果表明,FedRAN在通信、计算与标签受限条件下实现了高精度且资源高效的持续联邦学习。源码见https://github.com/JebacyrilArockiaraj/Fed-RAN-SSL。
原文摘要 · Abstract (English)
Federated continual learning (FCL) must learn from distributed task streams under limited resources, such as communication, computation, memory, and label availability. Existing FCL methods often rely on repeated local optimization, replay, and full supervision. Analytic alternatives avoid iterative training and replay, but using high-dimensional random features to improve accuracy requires a second-order feature statistic, the Gram matrix, which has a quadratic communication cost in the random feature size $M$. We propose FedRAN, a resource-aware analytic FCL framework that replaces gradient-based updates with compact random feature statistics. Each client transmits a truncated-SVD summary of its Gram matrix, reducing the dominant second-order upload from quadratic to linear in $M$ for fixed rank. The server performs a two-level QR-SVD subspace merge, spatially across clients and temporally across tasks, and solves a ridge classifier in closed form. FedRAN further supports label scarcity through prototype-based pseudo-labeling. Across CIFAR-100, ImageNet-R, and VTAB datasets, FedRAN improves average accuracy by up to 4.8 percentage points over the strongest baseline, uses 30.6-121.8$\times$ less per-client communication than optimization-based FCL, and is 190.3$\times$ faster on average than gradient-based baselines; with only 20% labels, pseudo-labeling improves average accuracy by up to 6.61 points. These results show that FedRAN enables accurate and resource-efficient FCL under communication, computation, and label constraints. The source code is available at https://github.com/JebacyrilArockiaraj/Fed-RAN-SSL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。