让边缘设备用大模型做任务，不装模型也能高效学习
Leveraging Foundation Models for Efficient Federated Learning in Resource-restricted Edge Networks
- 用联邦学习聚合设备知识，训练提示词生成器适配大模型
- 在5个图像数据集上表现优于基线方法
- 无需公共数据集，适合资源受限的物联网场景
最近，预训练的基础模型（FMs）与联邦学习（FL）结合，以在保护隐私的同时提升下游任务的训练效果。然而，在资源受限的物联网（IoT）设备上部署基础模型仍缺乏研究。本文提出一种新框架FedD2P，通过将边缘设备聚合的知识提炼给提示词生成器，实现对冻结的视觉-语言基础模型的高效适应，而无需在本地部署模型。该框架利用设备端的类别级局部知识和类别语义描述训练提示词生成器，避免对公共数据集的依赖。在CIFAR、OxfordPets、SVHN、EuroSAT和DTD等多个图像分类数据集上的实验表明，FedD2P在模型性能上优于现有基线方法。
原文摘要 · Abstract (English)
Recently pre-trained Foundation Models (FMs) have been combined with Federated Learning (FL) to improve training of downstream tasks while preserving privacy. However, deploying FMs over edge networks with resource-constrained Internet of Things (IoT) devices is under-explored. This paper proposes a novel framework, namely, Federated Distilling knowledge to Prompt (FedD2P), for leveraging the robust representation abilities of a vision-language FM without deploying it locally on edge devices. This framework distills the aggregated knowledge of IoT devices to a prompt generator to efficiently adapt the frozen FM for downstream tasks. To eliminate the dependency on a public dataset, our framework leverages perclass local knowledge from IoT devices and linguistic descriptions of classes to train the prompt generator. Our experiments on diverse image classification datasets CIFAR, OxfordPets, SVHN, EuroSAT, and DTD show that FedD2P outperforms the baselines in terms of model performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。