让大模型学会逐步推理,精准识别隐含恶意的图文混合内容
Can Thinking Models Think to Detect Hateful Memes?
- 用强化学习优化大模型的多模态推理链,提升判断逻辑
- 在基准测试中准确率与F1提升约1%,解释质量提高3%
- 适合研究有害内容检测、可解释AI的学者和工程师
仇恨表情包常需组合式多模态推理:图像或文本单独看可能无害,但二者结合传递有害意图。尽管基于思维链的多模态大模型(MLLMs)在视觉-语言理解上取得进展,其在仇恨表情包分析中的能力仍待探索。本文提出一种基于强化学习的后训练框架,通过任务特定奖励和新型组相对策略优化(GRPO)目标,提升思维型MLLM的推理能力。具体包括:(i) 对现成MLLMs在仇恨表情包理解上的表现进行系统实证研究;(ii) 通过知识蒸馏生成弱监督或伪监督的思维链推理,扩展现有仇恨表情包数据集;(iii) 引入基于GRPO的目标,联合优化分类性能与解释质量,促进细粒度、分步推理。在Hateful Memes基准测试中,该方法达到当前最优表现,准确率与F1提升约1%,解释质量提升约3%。代码、数据扩展及评估资源将公开发布以支持复现。
原文摘要 · Abstract (English)
Hateful memes often require compositional multimodal reasoning: the image and text may appear benign in isolation, yet their interaction conveys harmful intent. Although thinking-based multimodal large language models (MLLMs) have recently advanced vision-language understanding, their capabilities remain underexplored for hateful meme analysis. We propose a reinforcement learning based post-training framework that improves reasoning in thinking-based MLLMs through task-specific rewards and a novel Group Relative Policy Optimization (GRPO) objective. Specifically, we (i) conduct a systematic empirical study of off-the-shelf MLLMs for hateful meme understanding, (ii) extend an existing hateful meme dataset by generating weakly or pseudo-supervised chain-of-thought rationales via distillation, and (iii) introduce a GRPO-based objective that jointly optimizes meme classification and explanation quality to encourage fine-grained, step-by-step reasoning. Experiments on the Hateful Memes benchmark show that our approach achieves state-of-the-art performance, improving accuracy and F1 by approximately 1 percent and explanation quality by approximately 3 percent. We will publicly release our code, dataset extensions, and evaluation resources to support reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。