用大模型先弱标注,再精筛不确定样本,通信量减少八倍。
FAST: Federated Active Learning with Foundation Models for Communication-efficient Sampling and Training
- 两阶段流程:先用基础模型弱标注,再聚焦不确定样本精修
- 在5%标注预算下,准确率提升4.36%,通信轮次减少8倍
- 适合医疗等标注成本高、数据分散的场景
联邦主动学习(FAL)在保护数据隐私的同时利用分布式客户端的大量无标签数据,具有广阔前景。然而,真实应用受限于高昂的标注成本和通信密集的采样过程,尤其在跨域设置中,当客户端拥有大量本地数据时问题更显著。本文针对核心问题:如何在人工参与的学习中以最小标注代价降低通信开销?现有FAL方法通常采用迭代标注,将主动采样与联邦更新分离,导致多轮昂贵的通信与标注。为此,我们提出FAST,一种两阶段的FAL框架:第一阶段利用基础模型进行弱标注,第二阶段仅对最不确定样本进行精修。通过借助基础模型的表征知识,并将精修步骤整合进简化工作流,FAST显著降低了迭代主动采样带来的开销。在多种医学与自然图像基准上的实验表明,相比现有FAL方法,FAST平均提升4.36%准确率,且在有限5%标注预算下通信轮次减少八倍。
原文摘要 · Abstract (English)
Federated Active Learning (FAL) has emerged as a promising framework to leverage large quantities of unlabeled data across distributed clients while preserving data privacy. However, real-world deployments remain limited by high annotation costs and communication-intensive sampling processes, particularly in a cross-silo setting, when clients possess substantial local datasets. This paper addresses the crucial question: What is the best practice to reduce communication costs in human-in-the-loop learning with minimal annotator effort? Existing FAL methods typically rely on iterative annotation processes that separate active sampling from federated updates, leading to multiple rounds of expensive communication and annotation. In response, we introduce FAST, a two-pass FAL framework that harnesses foundation models for weak labeling in a preliminary pass, followed by a refinement pass focused exclusively on the most uncertain samples. By leveraging representation knowledge from foundation models and integrating refinement steps into a streamlined workflow, FAST substantially reduces the overhead incurred by iterative active sampling. Extensive experiments on diverse medical and natural image benchmarks demonstrate that FAST outperforms existing FAL methods by an average of 4.36% while reducing communication rounds eightfold under a limited 5% labeling budget.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。