用证据不确定性提升遥感图像分割的联邦微调可靠性
FedEU: Evidential Uncertainty-Driven Federated Fine-Tuning of Vision Foundation Models for Remote Sensing Image Segmentation
- 引入个性化证据不确定性建模,量化本地模型的认知差异
- 在三个异构数据集上实现更优分割性能,降低预测不确定性
- 适合关注联邦学习中模型可靠性与异构数据适应的开发者
在联邦环境中进行遥感图像分割(RSIS)受到越来越多关注,因其可在不共享原始影像或标注的情况下实现跨分布式数据集的协同训练。结合参数高效微调(PEFT)的联邦RSIS能够充分发挥预训练基础模型的泛化能力,同时最小化参数聚合和通信开销。然而,预训练模型对异构客户端数据的动态适配会增加更新不确定性,因缺乏对每个本地模型的不确定性估计,从而影响协作优化的可靠性。为此,我们提出FedEU,一种由证据不确定性驱动的联邦微调框架,用于视觉基础模型在遥感图像分割中的应用。具体而言,引入个性化证据不确定性建模,以量化本地模型的认知变异并识别局部数据分布下的高风险区域。此外,利用客户端特定特征嵌入(CFE)增强通道感知特征表示,通过个性化注意力和元素感知参数更新保留客户端特性。这些不确定性估计上传至服务器,通过基于Top-k的不确定性引导加权(TUW)策略实现自适应全局聚合,有效缓解分布偏移和不可靠更新的影响。在三个大规模异构数据集上的大量实验表明,FedEU表现优异。更重要的是,通过显式降低预测不确定性,FedEU实现了多样客户端间的均衡模型适应,带来更鲁棒可靠的联邦结果。代码将公开于 https://github.com/zxk688/FedEU。
原文摘要 · Abstract (English)
Remote sensing image segmentation (RSIS) in federated environments has gained increasing attention because it enables collaborative model training across distributed datasets without sharing raw imagery or annotations. Federated RSIS combined with parameter-efficient fine-tuning (PEFT) can unleash the generalization power of pretrained foundation models for real-world applications, with minimal parameter aggregation and communication overhead. However, the dynamic adaptation of pretrained models to heterogeneous client data inevitably increases update uncertainty and compromises the reliability of collaborative optimization due to the lack of uncertainty estimation for each local model. To bridge this gap, we present FedEU, a federated optimization framework for fine-tuning RSIS models driven by evidential uncertainty. Specifically, personalized evidential uncertainty modeling is introduced to quantify epistemic variations of local models and identify high-risk areas under local data distributions. Furthermore, the client-specific feature embedding (CFE) is exploited to enhance channel-aware feature representation while preserving client-specific properties through personalized attention and an element-aware parameter update approach. These uncertainty estimates are uploaded to the server to enable adaptive global aggregation via a Top-k uncertainty-guided weighting (TUW) strategy, which mitigates the impact of distribution shifts and unreliable updates. Extensive experiments on three large-scale heterogeneous datasets demonstrate the superior performance of FedEU. More importantly, FedEU enables balanced model adaptation across diverse clients by explicitly reducing prediction uncertainty, resulting in more robust and reliable federated outcomes. The source codes will be available at https://github.com/zxk688/FedEU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。