为大模型生成有害内容的概率提供可证明的安全下界
Sound Probabilistic Safety Bounds for Large Language Models
- 用隐空间特征优先探索可能产生有害输出的生成路径
- 在真实有害概率极低时仍能高效计算出有意义的下界
- 结果可形式化证明，适合用于大模型安全评估
我们提出一种新框架，用于计算大语言模型（LLM）对给定提示生成有害输出的概率上界。通过将Clopper-Pearson置信区间应用于该问题，获得可能近似正确（PAC）的边界。作为主要技术贡献，我们设计了一种算法，利用隐空间特征优先探索自回归生成树中更可能产生有害输出的分支。该方法尤其能在真实有害概率极小时高效计算出有用的下界，且所得下界具有保真性——即形式化证明其小于实际有害概率。实验结果表明，该方法能对主流LLM计算出非平凡的下界，首次实现大模型生成行为的统计认证与评估。
原文摘要 · Abstract (English)
We propose a novel framework for computing rigorous bounds on the probability that a large language model (LLM) generates harmful output to a given prompt. We study a new application of the Clopper-Pearson confidence intervals to obtain probably approximately correct (PAC) bounds for this problem. As our main technical contribution, we propose an algorithm that leverages features in the latent space to prioritize exploring branches in the auto-regressive generation tree that are more likely to produce harmful outputs. Our approach in particular enables the efficient computation of useful lower bounds, even in scenarios where the true harm probability is extremely small, and crucially, the obtained lower bounds are sound, i.e., formally proven to be less than the actual harmfulness probability: our experimental results demonstrate the effectiveness of our method by computing non-trivial lower bounds on state-of-the-art LLMs. This study newly enables the evaluation and statistical certification of LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。