通过缓存软标签和优化聚合，显著降低联邦学习通信开销。
Soft-Label Caching and Sharpening for Communication-Efficient Federated Distillation
- 缓存客户端软标签，避免重复传输，减少通信量
- 新聚合机制使模型在不同客户端下更稳定，准确率更高
- 适合资源受限场景下的高效联邦学习应用
联邦学习（FL）通过在本地保留数据实现分布式协作训练，提升隐私保护。传统方法依赖频繁参数交换，通信开销大且难以应对模型异构性。基于知识蒸馏的FL通过共享预测结果（软标签，即归一化概率分布）缓解此问题，但常出现多轮冗余传输，影响效率。本文提出SCARLET框架，融合同步软标签缓存与增强熵减聚合（Enhanced ERA）机制。该框架通过复用缓存的软标签，将通信成本降低高达50%，同时保持与现有方法相当的准确率。Enhanced ERA解决了传统温度调节聚合的固有不稳定性，确保在多样客户端环境下具备鲁棒控制与高性能。实验表明，SCARLET在准确率和通信效率上均优于当前最先进的蒸馏型联邦学习方法。代码已开源：https://github.com/kitsuyaazuma/SCARLET。
原文摘要 · Abstract (English)
Federated Learning (FL) enables collaborative model training across decentralized clients, enhancing privacy by keeping data local. Yet conventional FL, relying on frequent parameter-sharing, suffers from high communication overhead and limited model heterogeneity. Distillation-based FL approaches address these issues by sharing predictions (soft-labels, i.e., normalized probability distributions) instead, but they often involve redundant transmissions across communication rounds, reducing efficiency. We propose SCARLET, a novel framework integrating synchronized soft-label caching and an enhanced Entropy Reduction Aggregation (Enhanced ERA) mechanism. SCARLET minimizes redundant communication by reusing cached soft-labels, achieving up to 50% reduction in communication costs compared to existing methods while maintaining competitive accuracy. Enhanced ERA resolves the fundamental instability of conventional temperature-based aggregation, ensuring robust control and high performance in diverse client scenarios. Experimental evaluations demonstrate that SCARLET consistently outperforms state-of-the-art distillation-based FL methods in terms of accuracy and communication efficiency. The implementation of SCARLET is publicly available at https://github.com/kitsuyaazuma/SCARLET.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。