arXiv:2601.21796cs.CLcs.AI2026-01被引 1

让AI理解 meme 背后隐含的恶意,靠知识注入和双头学习提升检测精度。

KID: Knowledge-Injected Dual-Head Learning for Knowledge-Grounded Harmful Meme Detection

  • 用知识注入构建视觉与背景知识的推理链,引导模型理解隐性毒性。
  • 在五个多语言数据集上实现领先效果,性能提升2.1%至19.7%。
  • 适合需要精准识别文化隐喻中恶意内容的平台内容安全团队。

网络迷因已成为社交平台数字文化的重要载体,但其高度依赖隐喻与社会文化背景,常成为有害内容的隐蔽传播工具,给自动化内容审核带来挑战。现有方法多聚焦模态内与跨模态信号分析,却难以捕捉隐性毒性,因缺乏显式存在于迷因本身之外的背景知识。为此,本文提出KID(Knowledge-Injected Dual-Head Learning)框架,通过标签约束的知识蒸馏机制,将复杂的迷因理解分解为结构化推理链,明确关联视觉证据、背景知识与分类标签。该推理链在具体迷因上下文中嵌入外部知识,指导学习过程。同时,KID采用双头架构,联合优化语义生成与分类目标,实现语言推理对齐并保持稳定决策边界。在涵盖英语、中文及低资源孟加拉语的五个多语言数据集上进行的大量实验表明,KID在二分类与多标签有害迷因检测任务中均达到最新水平,主要评估指标上相较先前最佳方法提升2.1%至19.7%。消融实验证实知识注入与双头联合学习的有效性,二者贡献互补,显著增强模型鲁棒性与泛化能力。代码与数据已公开于https://github.com/PotatoDog1669/KID。

原文摘要 · Abstract (English)

Internet memes have become pervasive carriers of digital culture on social platforms. However, their heavy reliance on metaphors and sociocultural context also makes them subtle vehicles for harmful content, posing significant challenges for automated content moderation. Existing approaches primarily focus on intra-modal and inter-modal signal analysis, while the understanding of implicit toxicity often depends on background knowledge that is not explicitly present in the meme itself. To address this challenge, we propose KID, a Knowledge-Injected Dual-Head Learning framework for knowledge-grounded harmful meme detection. KID adopts a label-constrained distillation paradigm to decompose complex meme understanding into structured reasoning chains that explicitly link visual evidence, background knowledge, and classification labels. These chains guide the learning process by grounding external knowledge in meme-specific contexts. In addition, KID employs a dual-head architecture that jointly optimizes semantic generation and classification objectives, enabling aligned linguistic reasoning while maintaining stable decision boundaries. Extensive experiments on five multilingual datasets spanning English, Chinese, and low-resource Bengali demonstrate that KID achieves SOTA performance on both binary and multi-label harmful meme detection tasks, improving over previous best methods by 2.1%--19.7% across primary evaluation metrics. Ablation studies further confirm the effectiveness of knowledge injection and dual-head joint learning, highlighting their complementary contributions to robust and generalizable meme understanding. The code and data are available at https://github.com/PotatoDog1669/KID.

有害内容检测知识注入多语言双头学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。