提出统一框架，同时应对模态缺失与数据退化问题。
General Incomplete Multimodal Learning via Dynamic Quality Perception
- 用动态质量感知建模模态退化，实现自适应融合。
- 通过噪声注入学习噪声强度，提升质量评估可靠性。
- 适合处理真实场景中复杂缺失与退化数据的模型设计。
针对现实应用中多模态学习对缺失模态的鲁棒性需求，现有方法主要关注完整模态缺失（inter-modality missing），却忽视了模态虽存在但严重退化（intra-modality degradation）的情况。实践中两者常共现，导致现有方法失效。为此，本文提出通用不完整多模态学习（GIML）框架，通过动态质量感知统一处理两类问题。GIML将异构缺失模式建模为连续的模态信息退化，实现退化感知的自适应融合。为实现可靠的质量感知，引入噪声感知质量估计器，通过受控噪声注入学习从退化特征到噪声强度的映射。此外，提出噪声-语义解耦模块，分离语义信息与噪声干扰，增强对未见退化模式的鲁棒性与泛化能力。在包含多种模态类型的多个数据集上的大量实验验证了GIML的有效性与通用性。代码已开源：https://github.com/Yu-Five/GIML。
原文摘要 · Abstract (English)
Multimodal learning robust to missing modalities is essential for real-world applications. Existing methods mainly focus on inter-modality missing, where entire modalities are absent, while overlooking intra-modality degradation, where modalities are present but severely corrupted. In practice, these two types of missing often coexist, making existing approaches ineffective. To address this limitation, we propose General Incomplete Multimodal Learning (GIML), a unified framework that simultaneously handles both inter-modality missing and intra-modality degradation through dynamic quality perception. Specifically, GIML models heterogeneous missing patterns as continuous modality information degradation, enabling degradation-aware adaptive fusion. To achieve reliable quality perception, we introduce a Noise-aware Quality Estimator that learns the mapping from corrupted features to noise intensity through controlled noise injection. Furthermore, we propose a Noise-Semantic Decoupled module that separates semantic information from noise interference. This improves robustness and generalization to unseen corruption patterns. Extensive experiments across datasets with diverse modality types demonstrate the effectiveness and generality of GIML. Code is available at: https://github.com/Yu-Five/GIML.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。