让手机等设备自动学习新任务,还能省存储、防攻击。
SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devices
- 用编码器-解码器分离通用与任务特异性特征,减少存储占用。
- 自适应检测数据变化,自动判断是否需更新或启动防御。
- 适合资源受限设备,尤其在持续学习中应对恶意任务。
端设备的普及催生了分布式计算范式,设备本地模型需持续处理动态生成的数据,而数据漂移带来持续学习挑战。传统集中式持续学习因隐私和数据量问题不适用。为此,联邦持续学习(FCL)成为可行方案,可在保护用户数据的同时协同优化模型。针对资源受限设备中存储不足、任务切换检测依赖人工、难以应对新对抗性任务等问题,本文提出新型FCL框架SacFL。该框架采用编码器-解码器结构,分离任务鲁棒与敏感组件,仅保留轻量级敏感部分,显著降低存储需求。同时,利用对比学习实现自主数据漂移检测,可自动判断新任务是否出现及其性质(良性或恶意),无需额外信息即可触发持续学习或防御策略,更契合端设备实际需求。在Cifar100与THUCNews等多个文本和图像数据集上进行的全面实验验证了SacFL在类增量与域增量场景下的有效性。此外,已开发演示系统进一步证实其实用性。
原文摘要 · Abstract (English)
The proliferation of end devices has led to a distributed computing paradigm, wherein on-device machine learning models continuously process diverse data generated by these devices. The dynamic nature of this data, characterized by continuous changes or data drift, poses significant challenges for on-device models. To address this issue, continual learning (CL) is proposed, enabling machine learning models to incrementally update their knowledge and mitigate catastrophic forgetting. However, the traditional centralized approach to CL is unsuitable for end devices due to privacy and data volume concerns. In this context, federated continual learning (FCL) emerges as a promising solution, preserving user data locally while enhancing models through collaborative updates. Aiming at the challenges of limited storage resources for CL, poor autonomy in task shift detection, and difficulty in coping with new adversarial tasks in FCL scenario, we propose a novel FCL framework named SacFL. SacFL employs an Encoder-Decoder architecture to separate task-robust and task-sensitive components, significantly reducing storage demands by retaining lightweight task-sensitive components for resource-constrained end devices. Moreover, $\rm{SacFL}$ leverages contrastive learning to introduce an autonomous data shift detection mechanism, enabling it to discern whether a new task has emerged and whether it is a benign task. This capability ultimately allows the device to autonomously trigger CL or attack defense strategy without additional information, which is more practical for end devices. Comprehensive experiments conducted on multiple text and image datasets, such as Cifar100 and THUCNews, have validated the effectiveness of $\rm{SacFL}$ in both class-incremental and domain-incremental scenarios. Furthermore, a demo system has been developed to verify its practicality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。