构建首个情绪可控的多模态梗图重审基准,支持情感转化与结构保持。
MER-Bench: A Comprehensive Benchmark for Multimodal Meme Reappraisal
- 提出情绪可控制的多模态梗图重审任务,保留原场景与布局。
- 在真实数据集上测试发现现有模型在结构保真度上差距显著。
- 适合研究可控生成、情感感知与多模态编辑的研究者使用。
梗图是一种视觉与文本紧密结合的社会表达形式,共同传递微妙的情感与评论。受心理学认知重评启发,我们提出「梗图重审」这一新型多模态生成任务:将负面框架的梗图转化为建设性内容,同时保留其底层场景、实体和结构布局。不同于以往的梗图理解或生成工作,该任务需在多重语义与风格约束下实现情绪可控、结构保持的多模态变换。为此,我们构建了MER-Bench基准,包含真实世界梗图的细粒度多模态标注,涵盖源情感与目标情感、正面改写文本、视觉编辑说明及分类标签(视觉类型、情感极性、布局结构)。我们进一步提出基于多模态大模型作为裁判的结构化评估框架,分解为模态级生成质量、情绪可控性、结构保真度与全局情感对齐四项指标。跨代表性图像编辑与多模态生成系统的大量实验揭示了现有方法在结构保持、语义一致性和情感转换方面的显著不足。我们认为MER-Bench为可控梗图编辑与情感感知的多模态生成研究奠定了基础。代码已开源:https://github.com/one-seven17/MER-Bench。
原文摘要 · Abstract (English)
Memes represent a tightly coupled, multimodal form of social expression, in which visual context and overlaid text jointly convey nuanced affect and commentary. Inspired by cognitive reappraisal in psychology, we introduce Meme Reappraisal, a novel multimodal generation task that aims to transform negatively framed memes into constructive ones while preserving their underlying scenario, entities, and structural layout. Unlike prior works on meme understanding or generation, Meme Reappraisal requires emotion-controllable, structure-preserving multimodal transformation under multiple semantic and stylistic constraints. To support this task, we construct MER-Bench, a benchmark of real-world memes with fine-grained multimodal annotations, including source and target emotions, positively rewritten meme text, visual editing specifications, and taxonomy labels covering visual type, sentiment polarity, and layout structure. We further propose a structured evaluation framework based on a multimodal large language model (MLLM)-as-a-Judge paradigm, decomposing performance into modality-level generation quality, affect controllability, structural fidelity, and global affective alignment. Extensive experiments across representative image-editing and multimodal-generation systems reveal substantial gaps in satisfying the constraints of structural preservation, semantic consistency, and affective transformation. We believe MER-Bench establishes a foundation for research on controllable meme editing and emotion-aware multimodal generation. Our code is available at: https://github.com/one-seven17/MER-Bench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。