一次处理即可检测非法图像中的有害元素及其位置,精准且抗攻击。
Malicious Image Analysis via Vision-Language Segmentation Fusion: Detection, Element, and Location in One-shot
- 用视觉语言模型融合分割结果,实现端到端的有害内容定位。
- 在790张图像上达到85.8%元素召回率和92.1%分割成功率。
- 适合需要可解释性与高鲁棒性的内容审核系统使用。
检测非法视觉内容不仅需要图像级的NSFW标记,还需明确哪些物体导致违规以及其出现位置。本文提出一种零样本流水线,可一次性完成(i)判断图像是否含有害内容,(ii)识别每个关键元素,(iii)以像素级掩码精确定位这些元素。系统首先使用基础分割模型(SAM)生成候选对象掩码,并将其优化为独立区域;随后通过视觉语言模型(VLM)结合开放词汇提示对各区域评分,加权融合生成综合有害对象图。多分割器集成增强了对单一方法攻击的防御能力。在新标注的790张图像数据集(涵盖毒品、色情、暴力、极端主义内容)上,该方法达到85.8%元素级召回率、78.1%精确率和92.1%分割成功率,较直接零样本VLM定位提升27.4%召回率,精度相当。面对针对SAM和VLM设计的PGD对抗扰动,性能下降不超过10%,展现出强鲁棒性。整个流程可在数秒内完成,可无缝接入现有VLM工作流,是首个实现细粒度、可解释恶意图像审核的实用工具。
原文摘要 · Abstract (English)
Detecting illicit visual content demands more than image-level NSFW flags; moderators must also know what objects make an image illegal and where those objects occur. We introduce a zero-shot pipeline that simultaneously (i) detects if an image contains harmful content, (ii) identifies each critical element involved, and (iii) localizes those elements with pixel-accurate masks - all in one pass. The system first applies foundation segmentation model (SAM) to generate candidate object masks and refines them into larger independent regions. Each region is scored for malicious relevance by a vision-language model using open-vocabulary prompts; these scores weight a fusion step that produces a consolidated malicious object map. An ensemble across multiple segmenters hardens the pipeline against adaptive attacks that target any single segmentation method. Evaluated on a newly-annotated 790-image dataset spanning drug, sexual, violent and extremist content, our method attains 85.8% element-level recall, 78.1% precision and a 92.1% segment-success rate - exceeding direct zero-shot VLM localization by 27.4% recall at comparable precision. Against PGD adversarial perturbations crafted to break SAM and VLM, our method's precision and recall decreased by no more than 10%, demonstrating high robustness against attacks. The full pipeline processes an image in seconds, plugs seamlessly into existing VLM workflows, and constitutes the first practical tool for fine-grained, explainable malicious-image moderation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。