arXiv:2608.09240cs.LGcs.AI2026-08

解决多模态联邦学习中数据缺失问题,提升隐私保护下的模型性能。

Multimodal Federated Learning under Dual-Axis Modality Missingness

论文配图:Multimodal Federated Learning under Dual-Axis Modality Missingness
图 1 · 摘自论文原文
  • 根据模态缺失情况动态调整预测置信度,实现样本级自适应
  • 在四个数据集上平均宏F1提升1.6点,最高超基线2.2点
  • 适合医疗等隐私敏感场景,支持客户端本地个性化优化

多模态联邦学习在隐私敏感的健康感知和医疗场景中支持协同建模,但实际部署常面临双重模态缺失:客户端拥有的模态集合不同,且单个样本可能仅包含本地可用模态的部分。现有方法通常分别处理这两个维度。我们提出Flux框架,包含两个互补组件:首先,模态感知置信度调温通过掩码感知的单模态监督,为每个模态学习样本级置信度,并将已观测模态的置信度融合为样本自适应温度,根据证据质量和完整性调节预测锐度;其次,梯度解耦私有适配仅在客户端私有预测路径中应用该温度,而共享联邦模型仍采用标准无调温目标进行训练。这实现了样本级、客户端本地的置信度自适应,同时避免置信度相关梯度干扰共享表示学习。在四个多模态数据集上,Flux在每个数据集上的平均宏F1均最高,优于最强的特定数据集基线0.8~2.2点,平均提升1.6点。额外分析显示其具有良好的校准性,温度对模态缺失和输入扰动均敏感,且在仅私有路径调温下共享优化更稳定。代码已公开于https://github.com/AdibaOrz/Flux。

原文摘要 · Abstract (English)

Multimodal federated learning (FL) supports collaborative modeling in privacy-sensitive health-sensing and medical settings, but realistic deployments often exhibit dual-axis modality missingness: clients have different modality sets, and individual samples may contain only subsets of the modalities available locally. Existing methods typically address these two axes separately. We propose Flux, a multimodal federated learning framework built around two complementary components. First, modality-aware confidence tempering learns sample-specific confidence for each modality through mask-aware unimodal supervision and fuses the confidence estimates from observed modalities into a sample-adaptive temperature that adjusts predictive sharpness according to evidence quality and completeness. Second, gradient-decoupled private adaptation applies this temperature only to a client-private prediction pathway, while training the shared federated model with a standard, untempered objective. This enables sample-specific, client-local confidence adaptation without allowing confidence-dependent gradients to perturb shared representation learning. Across four multimodal datasets, Flux achieves the highest average macro-F1 on every dataset, outperforming the strongest dataset-specific baseline by 0.8~2.2 points and by 1.6 points on average. Additional analyses demonstrate favorable calibration, temperature sensitivity to both modality missingness and input corruption, and more stable shared optimization under private-only tempering. Our code is available at https://github.com/AdibaOrz/Flux.

联邦学习多模态隐私保护医疗AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。