通过合成数据与低秩适配，实现文本生成图像模型的安全对齐，可移除7倍更多有害概念。
AlignGuard: Scalable Safety Alignment for Text-to-Image Generation
- 用合成数据构建有害与安全图文对，结合自定义DPO训练安全专家。
- 通过新合并策略整合专家，使模型能同时屏蔽7倍于基线的有害概念。
- 方法可扩展性强，适合需高安全性生成的工业级应用。
文本到图像（T2I）模型广泛应用，但其有限的安全防护机制使用户面临有害内容风险，且易被滥用。现有安全措施多依赖文本过滤或概念移除，仅能消除少量特定概念。本文提出AlignGuard，一种T2I模型的安全对齐方法。通过合成生成包含有害与安全图像-文本对的数据集CoProV2，利用定制化直接偏好优化（DPO）训练以低秩适配（LoRA）形式存在的安全专家，引导生成避开特定安全相关概念。随后采用新型合并策略将多个专家融合为单一LoRA，实现最优可扩展性能。该专家式方法显著提升可扩展性，相比基线可移除7倍更多的有害概念。AlignGuard在多项基准测试中持续优于当前最优方法，并确立了T2I网络安全对齐的新范式。代码与数据将公开于https://safetydpo.github.io/。
原文摘要 · Abstract (English)
Text-to-image (T2I) models are widespread, but their limited safety guardrails expose end users to harmful content and potentially allow for model misuse. Current safety measures are typically limited to text-based filtering or concept removal strategies, able to remove just a few concepts from the model's generative capabilities. In this work, we introduce AlignGuard, a method for safety alignment of T2I models. We enable the application of Direct Preference Optimization (DPO) for safety purposes in T2I models by synthetically generating a dataset of harmful and safe image-text pairs, which we call CoProV2. Using a custom DPO strategy and this dataset, we train safety experts, in the form of low-rank adaptation (LoRA) matrices, able to guide the generation process away from specific safety-related concepts. Then, we merge the experts into a single LoRA using a novel merging strategy for optimal scaling performance. This expert-based approach enables scalability, allowing us to remove 7x more harmful concepts from T2I models compared to baselines. AlignGuard consistently outperforms the state-of-the-art on many benchmarks and establishes new practices for safety alignment in T2I networks. Code and data will be shared at https://safetydpo.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。