通过共识采样提升生成式AI安全性,自动规避不安全输出
Consensus Sampling for Safer Generative AI
- 用点对点中位数聚合多个概率分布,增强鲁棒性
- 在安全分布重叠时,风险低于最安全的s个分布平均值
- 适用于无法修改模型的场景,适合追求安全性的开发者
针对生成式AI中的不可检测风险,本文研究一个通用的鲁棒聚合问题:如何聚合多个概率分布以提升安全性。提出共识采样(consensus sampling),一种黑盒算法,在给定k个分布时,其风险与最安全的s个分布的平均风险相当,且在缺乏一致意见时选择不输出。该方法适用于可采样和评估输出概率的模型,提供与架构无关的安全保障。通过R-robustness形式化保证,同时控制信息泄露和对抗影响。受鲁棒统计学及Vyas等人(2023)可证明版权保护算法启发,表明标准混合易受单个不安全成分影响,而点对点中位数构造具备鲁棒性,所提高效采样器在最坏风险与拒答率之间达到帕累托最优。合成分布与图像生成实验验证了机制的有效性。方法要求安全分布间存在重叠,但能继承未知可靠子集的保证。
原文摘要 · Abstract (English)
Motivated by undetectable risks in generative AI, we study a general robust aggregation problem: how to aggregate several probability distributions to boost safety. We present consensus sampling, a black-box algorithm that, given k distributions, has risk competitive with the average risk of the safest $s$ while abstaining when there is insufficient agreement. This yields an architecture-agnostic approach to generative-model safety when the distributions are induced by models that can sample and evaluate output probabilities. We formalize the guarantee through R-robustness, which also bounds information leakage and adversarial influence. Inspired by robust statistics and the provable copyright protection algorithm of Vyas et al (2023), we show that while a standard mixture is vulnerable to one unsafe constituent, a pointwise-median construction provides robust intuition, and our efficient sampler is Pareto-optimal for the tradeoff between worst-case risk and abstention. Experiments on synthetic distributions and image generation illustrate the general mechanism and its motivating safety application. The method requires overlap among safe distributions, but it provides a model-agnostic way to inherit guarantees from an unknown reliable subset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。