用预训练大模型零样本判断图像安全,无需人工标注。
MLLM-as-a-Judge for Image Safety without Human Labeling
- 将安全规则显式化并简化条件链,提升模型判断准确性
- 通过去偏概率与分层推理,实现零样本下90%以上准确率
- 适合需要快速更新安全规则的AIGC内容审核场景
随着在线平台视觉内容激增,图像安全问题日益严峻。在人工智能生成内容(AIGC)背景下,许多图像生成模型可产出生殖或暴力等有害内容,亟需基于安全规则识别此类图像。预训练多模态大语言模型(MLLM)具备强大模式识别能力,但现有方法依赖人工标注数据微调,成本高且难以适应频繁变更的安全规则。本文探索在零样本设置下,仅通过查询预训练MLLM并依据预设安全宪章(一组安全规则)进行判断是否可行。研究发现直接查询效果不佳,原因包括规则主观性、规则文本冗长及模型固有偏见。为此提出新方法:对象化规则、评估规则与图像相关性、基于去偏标记概率做出快速判断,并在必要时采用级联思维链进行深度推理。实验表明该方法在零样本图像安全判断任务中表现优异,准确率达90%以上。
原文摘要 · Abstract (English)
Image content safety has become a significant challenge with the rise of visual media on online platforms. Meanwhile, in the age of AI-generated content (AIGC), many image generation models are capable of producing harmful content, such as images containing sexual or violent material. Thus, it becomes crucial to identify such unsafe images based on established safety rules. Pre-trained Multimodal Large Language Models (MLLMs) offer potential in this regard, given their strong pattern recognition abilities. Existing approaches typically fine-tune MLLMs with human-labeled datasets, which however brings a series of drawbacks. First, relying on human annotators to label data following intricate and detailed guidelines is both expensive and labor-intensive. Furthermore, users of safety judgment systems may need to frequently update safety rules, making fine-tuning on human-based annotation more challenging. This raises the research question: Can we detect unsafe images by querying MLLMs in a zero-shot setting using a predefined safety constitution (a set of safety rules)? Our research showed that simply querying pre-trained MLLMs does not yield satisfactory results. This lack of effectiveness stems from factors such as the subjectivity of safety rules, the complexity of lengthy constitutions, and the inherent biases in the models. To address these challenges, we propose a MLLM-based method includes objectifying safety rules, assessing the relevance between rules and images, making quick judgments based on debiased token probabilities with logically complete yet simplified precondition chains for safety rules, and conducting more in-depth reasoning with cascaded chain-of-thought processes if necessary. Experiment results demonstrate that our method is highly effective for zero-shot image safety judgment tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。