arXiv:2507.16854cs.CVcs.AI2025-07被引 1

CLAMP提升图文细粒度情感分析,有效对齐文本与图像局部区域。

CLAMP: Contrastive Learning with Adaptive Multi-loss and Progressive Fusion for Multimodal Aspect-Based Sentiment Analysis

  • 分阶段跨模态注意力融合,精准对齐文本词与图像局部区域。
  • 多任务对比学习增强跨模态表示一致性,提升细粒度情感判断准确率。
  • 动态加权损失机制缓解梯度冲突,适合多模态情感分析研究者。

多模态方面级情感分析(MABSA)旨在从图文配对数据中识别方面词并判断其细粒度情感极性,是提升产品评论系统和舆情监控效果的关键任务。现有方法存在跨模态对齐噪声和细粒度表示不一致等问题。全局模态对齐方法常忽略方面词与其对应局部视觉区域的关联,文本与图像间表示差距仍难弥合。为此,本文提出端到端的对比学习框架CLAMP,包含三个新模块:渐进式注意力融合网络、多任务对比学习和自适应多损失聚合。渐进式注意力融合网络通过分层多阶段跨模态交互,增强文本特征与图像区域的细粒度对齐,有效抑制无关视觉噪声。多任务对比学习结合全局模态对比与局部粒度对齐,提升跨模态表示一致性。自适应多损失聚合采用基于不确定性的动态权重机制,根据各任务不确定性调节损失贡献,缓解梯度干扰。在标准公开基准上的评估表明,CLAMP持续优于绝大多数现有最先进方法。

原文摘要 · Abstract (English)

Multimodal aspect-based sentiment analysis(MABSA) seeks to identify aspect terms within paired image-text data and determine their fine grained sentiment polarities, representing a fundamental task for improving the effectiveness of applications such as product review systems and public opinion monitoring. Existing methods face challenges such as cross modal alignment noise and insufficient consistency in fine-grained representations. While global modality alignment methods often overlook the connection between aspect terms and their corresponding local visual regions, bridging the representation gap between text and images remains a challenge. To address these limitations, this paper introduces an end to end Contrastive Learning framework with Adaptive Multi-loss and Progressive Attention Fusion(CLAMP). The framework is composed of three novel modules: Progressive Attention Fusion network, Multi-task Contrastive Learning, and Adaptive Multi-loss Aggregation. The Progressive Attention Fusion network enhances fine-grained alignment between textual features and image regions via hierarchical, multi-stage cross modal interactions, effectively suppressing irrelevant visual noise. Secondly, multi-task contrastive learning combines global modal contrast and local granularity alignment to enhance cross modal representation consistency. Adaptive Multi-loss Aggregation employs a dynamic uncertainty based weighting mechanism to calibrate loss contributions according to each task's uncertainty, thereby mitigating gradient interference. Evaluation on standard public benchmarks demonstrates that CLAMP consistently outperforms the vast majority of existing state of the art methods.

多模态情感分析对比学习图文对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。