arXiv:2608.16622cs.CVcs.AI2026-08

精准识别有害表情包攻击目标,提升内容安全检测细粒度。

HarmTrace: Anchor-Calibrated Decoupled Optimization for Fine-Grained Target Identification in Harmful Memes

论文配图:HarmTrace: Anchor-Calibrated Decoupled Optimization for Fine-Grained Target Identification in Harmful Memes
图 1 · 摘自论文原文
  • 通过锚点校准解耦优化,强化目标实体监督
  • 在Qwen3-VL-8B上将目标识别准确率从17.58%提升至52.51%
  • 适合关注多模态安全与细粒度内容理解的研究者

多模态有害表情包检测通常被定义为图像-文本有害性分类任务。模型可能正确判断有害性,却错误识别攻击目标或支持证据。因此,我们扩展该任务,引入细粒度目标识别,旨在确定攻击类型、具体对象及在表情包中的位置。模型需对每张表情包预测有害性,并对有害样本输出目标类别、目标实体、文本提及和视觉区域。为此,我们构建了Meme3W数据集,整合多个公开有害表情包数据集并提供人工验证的标注。同时提出联合记录准确率(JRA),要求有害性标签与所有目标识别字段同时正确才算成功。实验表明,主流多模态大模型在有害性准确率与JRA之间存在显著差距。为此,我们提出HarmTrace框架:先通过实体感知微调增强目标实体监督,再采用条件目标识别策略优化(CTPO)解耦有害性与目标识别优势,仅对有害样本中标签正确的响应进行目标识别优化;其中使用虚拟正向锚点(VPA)作为完全正确的参考标准进行优势归一化。HarmTrace在多个基线模型上均提升了JRA与有害性准确率,尤其在Qwen3-VL-8B上,JRA从17.58%提升至52.51%。

原文摘要 · Abstract (English)

Multimodal harmful meme detection is typically formulated as image--text harmfulness classification. A model may correctly predict harmfulness while misidentifying the attacked target or its supporting evidence. We therefore extend harmful meme detection with fine-grained target identification, asking what type of target is attacked, who is targeted, and where the target appears in the meme. The model predicts harmfulness for every meme and, for harmful memes, outputs the target category, target entity, textual mention, and visual region. To support this task, we introduce Meme3W, which unifies multiple public harmful meme datasets and provides human-verified annotations for harmful instances. We further introduce Joint Record Accuracy (JRA), a strict record-level metric requiring the harmfulness label and all target-identification fields to be jointly correct. Experiments with representative multimodal large language models reveal a substantial gap between harmfulness accuracy and JRA. To narrow this gap, we propose HarmTrace, an anchor-calibrated decoupled optimization framework. HarmTrace strengthens target-entity supervision through entity-aware supervised fine-tuning. It then applies Conditional Target-identification Policy Optimization (CTPO) to decouple harmfulness and target-identification advantages, restricting target-identification optimization to label-correct responses for harmful examples. CTPO uses a Virtual Positive Anchor (VPA) as a fully correct reference for target-identification advantage normalization. HarmTrace improves both JRA and harmfulness accuracy across the evaluated backbones, with JRA on the Qwen3-VL-8B backbone increasing from 17.58\% to 52.51\%. Our code is publicly available at https://github.com/llly1234/HarmTrace-for-Harmful-Memes.

有害检测多模态目标识别大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。