通过可调安全提示投影,实现生成图像安全与原提示行为的平衡。
SPOT: Selective Prompt Projection via Total Variation for Inference-Only Safe Text-to-Image Generation
- 基于总变差约束,在不重训练模型的前提下选择性修正提示词。
- 在四个数据集上降低14.2%至44.4%的不当生成风险,保持正常提示输出稳定。
- 适合需要高安全性和推理阶段快速干预的图像生成应用。
文本到图像扩散模型虽能生成高质量内容,但实际使用中需抑制不安全输出,同时保留对良性提示的行为。本文以冻结生成器的提示条件分布为基准,利用总变差(TV)限制预期风险的变动范围,提出安全-提示对齐权衡(SPAT)概念。定义参考风险不超过阈值tau的提示集合为tau安全集,将干预视为向该集合中邻近提示的投影。提出无需重训练或学习类别特定重写器的推理时框架SPOT,通过LLM排序候选重写,用VLM在相同tau下验证生成图像。在四个数据集和三种扩散模型上,相比强基线,SPOT实现14.2%~44.4%的不当生成(IP)分数下降,同时保持对良性提示的输出接近原始参考分布。
原文摘要 · Abstract (English)
Text-to-Image (T2I) diffusion models enable high quality open ended synthesis, but practical use requires suppressing unsafe generations while preserving behavior on benign prompts. We study this tension relative to the frozen generator, using its prompt conditioned distribution as the preservation reference. Since T2I safety is commonly evaluated by bounded risk scores on generated images, total variation (TV) bounds how much expected risk can change from this reference. We call this fixed reference constraint the Safety-Prompt Alignment Tradeoff (SPAT): reducing expected unsafety requires prompt conditioned distributional deviation. To make this deviation selective and adjustable, we define the tau safe set as prompts whose reference risk is at most tau, and cast intervention as projection toward nearby prompts in this set. We propose Selective Prompt prOjecTion (SPOT), an inference time framework that approximates this projection without retraining the generator or learning a category specific rewriter. SPOT uses an LLM to rank candidate rewrites and a safeguard VLM to accept generated images under the same tau. Across four datasets and three diffusion backbones, SPOT achieves relative inappropriate (IP) score reductions from 14.2% to 44.4% over strong safety alignment baselines while keeping benign prompt behavior close to the fixed reference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。