用12个视角分析梗图中的幽默与仇恨,让AI判断更准且能讲清楚理由。
Beyond a Joke: Multi-Angle Reasoning for Detecting and Explaining Harmful Humor in Memes

- 从幽默和仇恨理论出发设计12种分析视角
- 在双数据集上幽默识别达80.3%,仇恨识别75.9%
- 解释内容结合视角权重,适合需要透明判断的场景
网络梗图融合视觉线索、文本内容与文化语境,当幽默、讽刺与有害意图共存时,理解难度极大。现有多模态分类器或忽略这些相互关联,或仅提供有限可解释性。本文提出MAR-12框架,利用视觉语言模型(VLMs)在幽默与仇恨并存场景下实现梗图检测与理解。该框架首先基于幽默与仇恨理论构建12种结构化视角解析每张梗图,再通过角色感知的软门控注意力机制学习各视角贡献度,最后采用原型分类器完成预测。解释信息由视角特异性推理与学习到的注意力权重共同生成,确保透明且上下文相关的理由。在PrideMM与Memotion数据集上的实验显示,其幽默检测准确率达80.3%,仇恨检测达75.9%,优于现有方法。人类与GPT-4评估均证实,MAR-12生成的解释在幽默与有害线索交织的案例中尤为连贯且有说服力。
原文摘要 · Abstract (English)
Internet memes intertwine visual cues, textual content, and cultural context, making them particularly challenging to interpret in scenarios where humor, sarcasm, and harmful intent coexist. These complexities highlight the need for explainable meme understanding systems that can provide reliable and structured reasoning to support both accurate classification and human interpretability. However, existing multimodal classifiers either overlook these interdependencies or provide only limited interpretability. In this paper, we introduce MAR-12, a novel framework that leverages Vision Language Models (VLMs) for meme detection and understanding in settings where humorous and hateful elements may coexist. The framework first interprets each meme through twelve structured perspectives derived from humor and hate theories. It then applies a role-aware soft-gated attention mechanism to learn how much each perspective should contribute, followed by a prototype-based classifier for the final prediction. Finally, explanations are synthesized using both perspective-specific reasoning and learned attention weights, ensuring transparent and context-grounded justifications. We evaluate MAR-12 on the PrideMM and Memotion datasets, where it achieves up to 80.3% accuracy for humor detection and 75.9% accuracy for hate detection, outperforming state-of-the-art approaches. Furthermore, both human and GPT-4-based evaluations confirm that MAR-12 produces coherent and persuasive explanations, particularly for memes in which humorous and harmful cues co-occur.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。