让多模态大模型学会删掉特定关系带来的安全隐患
Relationship-Aware Safety Unlearning for Multimodal LLMs
- 用关系三元组识别不安全的物体-动作-物体组合
- 仅修改相关参数，避免误伤正常用法
- 适合需要精准安全控制的生成模型应用
生成式多模态模型可能因关系而产生安全问题：两个本无害的概念在特定动作或关系下会变得危险（如儿童饮酒）。现有去学习和概念擦除方法通常针对孤立概念或图文对，易导致对相同物体或关系的良性使用造成附带损伤。我们提出关系感知的安全去学习框架，显式表示不安全的物体-关系-物体（O-R-O）三元组，并通过参数高效的LoRA方法精准编辑，抑制不安全三元组，同时保持物体边缘分布和安全邻近关系不变。实验基于CLIP模型，在改写、上下文和分布外图像攻击下评估了鲁棒性。
原文摘要 · Abstract (English)
Generative multimodal models can exhibit safety failures that are inherently relational: two benign concepts can become unsafe when linked by a specific action or relation (e.g., child-drinking-wine). Existing unlearning and concept-erasure approaches often target isolated concepts or image-text pairs, which can cause collateral damage to benign uses of the same objects and relations. We propose relationship-aware safety unlearning: a framework that explicitly represents unsafe object-relation-object (O-R-O) tuples and applies targeted parameter-efficient edits (LoRA) to suppress unsafe tuples while preserving object marginals and safe neighboring relations. We include CLIP-based experiments and robustness evaluation under paraphrase, contextual, and out-of-distribution image attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。