提出动态感知不确定性的学习框架,提升模型在多模态场景下的鲁棒性。
DUAL: Dynamic Uncertainty-Aware Learning
- 通过动态建模特征不确定性,结合特征与学习过程实时优化估计。
- 在多个数据集上显著提升准确率,最高达7.1%(CIFAR-10)。
- 适合处理含噪声或异构输入的复杂多模态任务,如情感分析。
深度学习模型在多种学习场景中常面临特征不确定性问题,严重影响其性能与可靠性,尤其在多模态场景下,需融合来自不同源、具有内在不确定性的信息。本文提出统一框架DUAL(Dynamic Uncertainty-Aware Learning),有效处理单模态与多模态场景中的特征不确定性。DUAL引入三项核心创新:动态特征不确定性建模,通过联合考虑特征特性与学习动态持续优化不确定性估计;自适应分布感知调制,通过动态调整样本影响以维持特征分布平衡;不确定性感知的跨模态关系学习,显式建模跨模态交互中的不确定性。大量实验表明,DUAL在多个领域均表现优异:在计算机视觉任务中,于CIFAR-10、CIFAR-100和Tiny-ImageNet上分别取得7.1%、6.5%和2.3%的准确率提升;在多模态学习中,于CMU-MOSEI和CMU-MOSI的情感分析任务上分别获得4.1%和2.8%的增益,于MISR任务中提升1.4%。代码即将开源。
原文摘要 · Abstract (English)
Deep learning models frequently encounter feature uncertainty in diverse learning scenarios, significantly impacting their performance and reliability. This challenge is particularly complex in multi-modal scenarios, where models must integrate information from different sources with inherent uncertainties. We propose Dynamic Uncertainty-Aware Learning (DUAL), a unified framework that effectively handles feature uncertainty in both single-modal and multi-modal scenarios. DUAL introduces three key innovations: Dynamic Feature Uncertainty Modeling, which continuously refines uncertainty estimates through joint consideration of feature characteristics and learning dynamics; Adaptive Distribution-Aware Modulation, which maintains balanced feature distributions through dynamic sample influence adjustment; and Uncertainty-aware Cross-Modal Relationship Learning, which explicitly models uncertainties in cross-modal interactions. Through extensive experiments, we demonstrate DUAL's effectiveness across multiple domains: in computer vision tasks, it achieves substantial improvements of 7.1% accuracy on CIFAR-10, 6.5% accuracy on CIFAR-100, and 2.3% accuracy on Tiny-ImageNet; in multi-modal learning, it demonstrates consistent gains of 4.1% accuracy on CMU-MOSEI and 2.8% accuracy on CMU-MOSI for sentiment analysis, while achieving 1.4% accuracy improvements on MISR. The code will be available on GitHub soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。