arXiv:2503.04151cs.CVcs.AI2025-03ICCV被引 14

提出一种鲁棒多视图学习方法,融合视图特征并对抗数据噪声。

Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbation

  • 通过样本级注意力融合多视图特征,生成统一表示。
  • 模拟噪声和不可用扰动,用对比学习对齐不同表示以增强鲁棒性。
  • 无需标签,适用于聚类、分类和跨模态检索,可即插即用。

近年来,多视图学习(MVL)因其能融合多视角的判别信息而受到广泛关注。然而,真实世界中的多视图数据常具有异构性和不完美性,导致针对特定视图组合设计的MVL方法泛化能力差且效果受限。为此,我们提出一种新颖的鲁棒多视图学习方法(RML),同时实现表示融合与对齐。具体地,我们设计了一个简单的多视图变换器融合网络,将异构多视图数据转换为同质词嵌入,并通过样本级注意力机制整合多视图信息,获得融合表示。此外,我们提出基于模拟扰动的多视图对比学习框架,动态生成噪声和不可用扰动以模拟不完美数据条件。模拟出的噪声与不可用数据分别生成两个不同融合表示,利用对比学习使二者对齐,从而学习判别性强且鲁棒的表示。RML为自监督方法,亦可作为正则化用于下游任务。实验中,我们在多视图无监督聚类、噪声标签分类以及跨模态哈希检索中验证其有效性。大量对比实验与消融研究证明了RML的有效性。代码已公开于https://github.com/SubmissionsIn/RML。

原文摘要 · Abstract (English)

Recently, multi-view learning (MVL) has garnered significant attention due to its ability to fuse discriminative information from multiple views. However, real-world multi-view datasets are often heterogeneous and imperfect, which usually causes MVL methods designed for specific combinations of views to lack application potential and limits their effectiveness. To address this issue, we propose a novel robust MVL method (namely RML) with simultaneous representation fusion and alignment. Specifically, we introduce a simple yet effective multi-view transformer fusion network where we transform heterogeneous multi-view data into homogeneous word embeddings, and then integrate multiple views by the sample-level attention mechanism to obtain a fused representation. Furthermore, we propose a simulated perturbation based multi-view contrastive learning framework that dynamically generates the noise and unusable perturbations for simulating imperfect data conditions. The simulated noisy and unusable data obtain two distinct fused representations, and we utilize contrastive learning to align them for learning discriminative and robust representations. Our RML is self-supervised and can also be applied for downstream tasks as a regularization. In experiments, we employ it in multi-view unsupervised clustering, noise-label classification, and as a plug-and-play module for cross-modal hashing retrieval. Extensive comparison experiments and ablation studies validate RML's effectiveness. Code is available at https://github.com/SubmissionsIn/RML.

多视图学习对比学习鲁棒性自监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。