arXiv:2511.08152cs.CVcs.LG2025-11AAAI被引 1

提出平衡多目标优化的跨模态域适应方法,解决不同模态间域偏移不一致问题。

Boomda: Balanced Multi-objective Optimization for Multimodal Domain Adaptation

  • 分模态独立学习表示,再通过相关性对齐实现源目标域匹配
  • 构建多目标优化框架,求解帕累托最优解,提升整体适配效果
  • 算法高效可闭式求解,适合多模态数据少标注场景

多模态学习虽在多个领域取得成功,但面临标注成本过高的挑战。为缓解标注数据稀缺问题,无监督域适应被广泛采用,但现有研究多集中于单模态,多模态场景仍缺乏深入探索。本文研究异构多模态域适应,核心挑战在于不同模态从源域到目标域存在差异化的域偏移。我们首先引入信息瓶颈法独立学习各模态表示,再在表示空间中通过相关性对齐实现源-目标域匹配。为平衡所有模态的域对齐,将问题建模为多目标优化任务,寻求帕累托最优解。基于模型特性,问题可简化为二次规划,进一步近似得到闭式解,从而设计出高效且平衡的多模态域适应算法——Boomda。大量实验证明其有效性,优于现有方法。代码已开源:https://github.com/sunjunaimer/Boomda.git。

原文摘要 · Abstract (English)

Multimodal learning, while contributing to numerous success stories across various fields, faces the challenge of prohibitively expensive manual annotation. To address the scarcity of annotated data, a popular solution is unsupervised domain adaptation, which has been extensively studied in unimodal settings yet remains less explored in multimodal settings. In this paper, we investigate heterogeneous multimodal domain adaptation, where the primary challenge is the varying domain shifts of different modalities from the source to the target domain. We first introduce the information bottleneck method to learn representations for each modality independently, and then match the source and target domains in the representation space with correlation alignment. To balance the domain alignment of all modalities, we formulate the problem as a multi-objective task, aiming for a Pareto optimal solution. By exploiting the properties specific to our model, the problem can be simplified to a quadratic programming problem. Further approximation yields a closed-form solution, leading to an efficient modality-balanced multimodal domain adaptation algorithm. The proposed method features \textbf{B}alanced multi-\textbf{o}bjective \textbf{o}ptimization for \textbf{m}ultimodal \textbf{d}omain \textbf{a}daptation, termed \textbf{Boomda}. Extensive empirical results showcase the effectiveness of the proposed approach and demonstrate that Boomda outperforms the competing schemes. The code is is available at: https://github.com/sunjunaimer/Boomda.git.

多模态域适应优化算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。