通过原型对齐与分层注意力融合，提升多模态意图识别的鲁棒性。
MVCL-DAF++: Enhancing Multimodal Intent Recognition via Prototype-Aware Contrastive Alignment and Coarse-to-Fine Dynamic Attention Fusion
- 用原型引导实例对齐，增强语义一致性。
- 在两个数据集上罕见类识别准确率分别提升4.18%和1.05%。
- 适合需要高鲁棒性的多模态交互系统使用。
多模态意图识别（MMIR）面临语义定位弱、噪声或稀有类别下鲁棒性差的问题。本文提出MVCL-DAF++，在原有基础上引入两个核心模块：(1) 原型感知对比对齐，将样本与类别级原型对齐，提升语义一致性；(2) 粗粒度到细粒度注意力融合，整合全局模态摘要与令牌级特征，实现层级化跨模态交互。在MIntRec和MIntRec2.0数据集上，MVCL-DAF++达到新最优性能，稀有类别识别的加权F1值分别提升+4.18%和+1.05%。结果验证了原型引导学习与分层融合在提升多模态理解鲁棒性方面的有效性。代码已开源：https://github.com/chr1s623/MVCL-DAF-PlusPlus。
原文摘要 · Abstract (English)
Multimodal intent recognition (MMIR) suffers from weak semantic grounding and poor robustness under noisy or rare-class conditions. We propose MVCL-DAF++, which extends MVCL-DAF with two key modules: (1) Prototype-aware contrastive alignment, aligning instances to class-level prototypes to enhance semantic consistency; and (2) Coarse-to-fine attention fusion, integrating global modality summaries with token-level features for hierarchical cross-modal interaction. On MIntRec and MIntRec2.0, MVCL-DAF++ achieves new state-of-the-art results, improving rare-class recognition by +1.05\% and +4.18\% WF1, respectively. These results demonstrate the effectiveness of prototype-guided learning and coarse-to-fine fusion for robust multimodal understanding. The source code is available at https://github.com/chr1s623/MVCL-DAF-PlusPlus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。