用知识蒸馏替代梯度通信，让物联网设备高效训练深层模型。
RIFLE: Robust Distillation-based FL for Deep Model Deployment on Resource-Constrained IoT Networks
- 以输出概率代替梯度传输，降低通信开销和隐私泄露风险。
- 在非独立同分布数据下，误报率降低87.5%，抗攻击能力提升62.5%。
- 可在0.3 GFLOPS设备上将VGG19训练时间从600天缩短至1.39小时。
联邦学习（FL）是资源受限物联网（IoT）环境中的主流分布式学习范式。这些设备通常依赖轻量级模型（TinyML），通过与中心服务器共享梯度来协同训练全局模型，同时保护数据隐私。然而，随着数据异构性和任务复杂性增加，小型模型难以捕捉复杂模式，尤其在极端非独立同分布（non-IID）条件下表现不佳。此外，抵御恶意客户端和污染更新仍是重大挑战。为此，本文提出RIFLE——一种基于知识蒸馏的鲁棒联邦学习框架，将梯度共享替换为基于logit的知识迁移。通过知识蒸馏聚合机制，RIFLE实现了在资源受限的物联网系统中训练VGG-19和ResNet18等深层模型。进一步地，基于KL散度的验证机制可量化客户端更新的可靠性，无需暴露原始数据，实现高可信度与隐私保护并存。在三个基准数据集（MNIST、CIFAR-10、CIFAR-100）上的实验表明，RIFLE在异构non-IID条件下将误报率降低最高达87.5%，中毒攻击缓解能力提升62.5%，且在仅10轮内准确率比传统方法最高提升28.3%。值得注意的是，该框架将VGG19在典型物联网设备（0.3 GFLOPS）上的训练时间从超过600天压缩至仅1.39小时，使深度学习在资源受限网络中成为可能。
原文摘要 · Abstract (English)
Federated learning (FL) is a decentralized learning paradigm widely adopted in resource-constrained Internet of Things (IoT) environments. These devices, typically relying on TinyML models, collaboratively train global models by sharing gradients with a central server while preserving data privacy. However, as data heterogeneity and task complexity increase, TinyML models often become insufficient to capture intricate patterns, especially under extreme non-IID (non-independent and identically distributed) conditions. Moreover, ensuring robustness against malicious clients and poisoned updates remains a major challenge. Accordingly, this paper introduces RIFLE - a Robust, distillation-based Federated Learning framework that replaces gradient sharing with logit-based knowledge transfer. By leveraging a knowledge distillation aggregation scheme, RIFLE enables the training of deep models such as VGG-19 and Resnet18 within constrained IoT systems. Furthermore, a Kullback-Leibler (KL) divergence-based validation mechanism quantifies the reliability of client updates without exposing raw data, achieving high trust and privacy preservation simultaneously. Experiments on three benchmark datasets (MNIST, CIFAR-10, and CIFAR-100) under heterogeneous non-IID conditions demonstrate that RIFLE reduces false-positive detections by up to 87.5%, enhances poisoning attack mitigation by 62.5%, and achieves up to 28.3% higher accuracy compared to conventional federated learning baselines within only 10 rounds. Notably, RIFLE reduces VGG19 training time from over 600 days to just 1.39 hours on typical IoT devices (0.3 GFLOPS), making deep learning practical in resource-constrained networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。