arXiv:2608.04791cs.CV2026-08中稿 · the SPIE Artificia…

对比四种视觉语言模型适配策略,为遥感联邦学习选型提供依据。

On the Effectiveness of Adaptation Strategies for VLM-Based Federated Learning in Remote Sensing

论文配图:On the Effectiveness of Adaptation Strategies for VLM-Based Federated Learning in Remote Sensing
图 1 · 摘自论文原文
  • 比较全微调、编码器微调、提示学习和低秩适配在遥感图像分类中的表现。
  • 发现不同策略在泛化能力、通信开销和计算复杂度间存在显著权衡。
  • 给出针对不同资源约束的适配策略选择指南,适合遥感与联邦学习研究者。

联邦学习(FL)使分布式图像数据集上的深度学习模型协同训练成为可能,无需集中数据。该范式在遥感(RS)领域尤为重要,因法律、隐私及带宽限制阻碍了数据共享。然而,客户端间训练数据异构性(非独立同分布数据)会阻碍收敛并降低全局模型泛化能力。为缓解此问题,可利用具有可迁移表征的视觉语言模型(VLM),其在分布偏移下表现出鲁棒性。但其大参数量会显著增加联邦环境中的通信开销和本地计算复杂度。因此,需选择兼顾泛化能力与通信/计算约束的适配策略。本文首次对遥感图像分类中基于VLM的联邦学习适配策略进行系统比较,考察全微调、编码器特定微调、提示学习和低秩适配(LoRA)三种指标:1)非独立同分布数据下的泛化能力;2)通信开销;3)本地计算复杂度。在BigEarthNet-S2、EuroSAT、RESISC45和ImageNet上实验显示,各策略在任务特化、跨域泛化与效率之间存在明显权衡。基于结果,我们提出适用于不同运行约束的适配策略选择指南。代码已公开于https://git.tu-berlin.de/rsim/FL-RS-VLM。

原文摘要 · Abstract (English)

Federated learning (FL) enables collaborative training of deep learning models across decentralized image archives without requiring data centralization. This paradigm is particularly relevant in remote sensing (RS), where legal regulations, privacy concerns, and bandwidth constraints restrict data sharing. However, the presence of training data heterogeneity across clients (known as non-IID data) can impede convergence and limit the generalization capability of the aggregated global model. To mitigate the adverse effects of training data heterogeneity, vision-language models (VLMs) can be leveraged in FL due to their transferable representations, which have demonstrated robustness under distribution shifts. However, their large parameter size may substantially increase communication overhead and local computational complexity in federated settings. Therefore, it is crucial to select an appropriate VLM adaptation strategy that balances the generalization ability with the communication and computational constraints. To address this issue, in this paper, we present the first comparative study of VLM adaptation strategies for FL in the context of RS image classification. We investigate full fine-tuning, encoder-specific fine-tuning, prompt learning, and low-rank adaptation (LoRA) tuning, and analyze them with respect to three criteria: 1) generalization capability under non-IID data, 2) communication overhead, and 3) local computational complexity. Experiments on BigEarthNet-S2, EuroSAT, RESISC45, and ImageNet reveal distinct trade-offs between task specialization, cross-domain generalization, and efficiency. Based on our findings, we derive a guideline for the selection of an appropriate VLM adaptation strategy in FL for RS image classification under different operational constraints. The code of this work is publicly available at https://git.tu-berlin.de/rsim/FL-RS-VLM.

联邦学习视觉语言模型遥感图像模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。