arXiv:2511.12554cs.CV2025-11被引 5

构建可解释的视觉情绪数据集,让模型看清图像如何引发情绪

EmoVerse: A MLLMs-Driven Emotion Representation Dataset for Interpretable Visual Emotion Analysis

  • 用三元组分解情绪,定位图像中每个元素的作用
  • 覆盖219,000张图,支持离散与连续情绪表示双标注
  • 提供可视化归因解释,适合可解释性研究者使用

视觉情绪分析(VEA)旨在弥合视觉内容与人类情绪反应之间的情感鸿沟。尽管前景广阔,该领域进展仍受限于缺乏开源且可解释的数据集。现有研究多为整图赋予单一离散情绪标签,难以揭示视觉元素如何影响情绪。本文提出EmoVerse,一个大规模开源数据集,通过受知识图谱启发的多层标注,实现可解释的视觉情绪分析。将情绪分解为背景-属性-主体(B-A-S)三元组,并将每个元素关联到具体视觉区域,实现词级与主体级的情绪推理。数据集包含超过219,000张图像,支持类别情绪状态(CES)与维度情绪空间(DES)双重标注,兼顾离散与连续情绪表征。采用新颖的多阶段流程,以最小人力成本确保标注可靠性。此外,提出一种可解释模型,将视觉线索映射至DES表示并生成详细归因解释。数据集、标注流程与模型共同构成可解释高阶情绪理解的完整基础。

原文摘要 · Abstract (English)

Visual Emotion Analysis (VEA) aims to bridge the affective gap between visual content and human emotional responses. Despite its promise, progress in this field remains limited by the lack of open-source and interpretable datasets. Most existing studies assign a single discrete emotion label to an entire image, offering limited insight into how visual elements contribute to emotion. In this work, we introduce EmoVerse, a large-scale open-source dataset that enables interpretable visual emotion analysis through multi-layered, knowledge-graph-inspired annotations. By decomposing emotions into Background-Attribute-Subject (B-A-S) triplets and grounding each element to visual regions, EmoVerse provides word-level and subject-level emotional reasoning. With over 219k images, the dataset further includes dual annotations in Categorical Emotion States (CES) and Dimensional Emotion Space (DES), facilitating unified discrete and continuous emotion representation. A novel multi-stage pipeline ensures high annotation reliability with minimal human effort. Finally, we introduce an interpretable model that maps visual cues into DES representations and provides detailed attribution explanations. Together, the dataset, pipeline, and model form a comprehensive foundation for advancing explainable high-level emotion understanding.

视觉情绪可解释性多模态数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。