arXiv:2506.08477cs.CL2025-06中稿 · ACM Web Conference…被引 5

用文字描述让小模型看懂图片,实现低成本可解释的有害表情包检测。

Read as You See: Guiding Unimodal LLMs for Low-Resource Explainable Harmful Meme Detection

  • 通过文本转换还原图片关键信息,让小语言模型‘读图’
  • 零样本链式思考提示使检测结果可解释,性能媲美大模型
  • 适合资源有限、需透明决策的内容审核场景

有害表情包检测对维护网络环境安全至关重要,但现有方法往往资源消耗大、灵活性差且缺乏可解释性,限制了其在真实内容审核中的应用。本文提出U-CoT+,一种资源高效框架,通过充分利用轻量级单模态大语言模型(LLM)能力,优先保障可访问性、灵活性与透明性。不直接调用多模态大模型作为黑箱分类器,而是通过高保真图文转换管道,将多模态表情包转化为保留关键视觉信息的自然语言描述,使仅处理文本的LLM能‘阅读’图片。基于生成的文本输入,采用人工设计的上下文感知、可解释的零样本链式思考(CoT)提示,引导单模态LLM进行推理,提供可问责的分步解释,并灵活适配不同社会文化标准。在七个基准数据集上的实验表明,U-CoT+性能接近资源密集型基线,验证了其作为可扩展、可解释、低资源解决方案的潜力。

原文摘要 · Abstract (English)

Detecting harmful memes is crucial for safeguarding the integrity and harmony of online environments, yet existing detection methods are often resource-intensive, inflexible, and lacking explainability, limiting their applicability in assisting real-world web content moderation. We propose U-CoT+, a resource-efficient framework that prioritizes accessibility, flexibility and transparency in harmful meme detection by fully harnessing the capabilities of lightweight unimodal large language models (LLMs). Instead of directly prompting or fine-tuning large multimodal models (LMMs) as black-box classifiers, we avoid immediate reasoning over complex visual inputs but decouple meme content recognition from meme harmfulness analysis through a high-fidelity meme-to-text pipeline, which collaborates lightweight LMMs and LLMs to convert multimodal memes into natural language descriptions that preserve critical visual information, thus enabling text-only LLMs to "see" memes by "reading". Grounded in textual inputs, we further guide unimodal LLMs' reasoning under zero-shot Chain-of-Thoughts (CoT) prompting with targeted, interpretable, context-aware, and easily obtained human-crafted guidelines, thus providing accountable step-by-step rationales, while enabling flexible and efficient adaptation to diverse sociocultural criteria of harmfulness. Extensive experiments on seven benchmark datasets show that U-CoT+ achieves performance comparable to resource-intensive baselines, highlighting its effectiveness and potential as a scalable, explainable, and low-resource solution to support harmful meme detection.

表情包检测可解释性轻量化模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。