首次为生成式语言模型提供可认证的抗污染攻击防护框架。
Towards Poisoning Robustness Certification for Natural Language Generation
- 提出TPA算法,计算诱导有害输出所需的最小污染预算。
- 实测在0.5%数据污染下仍可认证工具调用安全性,支持8个词的稳定生成期。
- 适用于高安全场景,如金融、医疗等关键领域的模型部署。
理解自然语言生成的可靠性对在安全敏感领域部署基础模型至关重要。尽管已有的认证污染防御能为分类任务提供可证明的鲁棒性边界,但其无法应对自回归生成:既无法处理序列预测,也无法应对语言模型指数级庞大的输出空间。为此,我们形式化了两个安全属性:稳定性(对任意生成变更的鲁棒性)与有效性(对特定有害变更的鲁棒性)。我们提出首个可认证有效性的算法——目标分区聚合(TPA),通过计算引发特定有害类别、词或短语所需的最小污染预算来实现。进一步地,我们利用混合整数线性规划(MILP)扩展TPA,以获得多轮生成的更紧致保证。实验表明,TPA在多种设置中均有效:在对手修改最多0.5%数据集时,仍可认证代理工具调用的有效性;并在基于偏好对齐的任务中,验证了8个词的稳定性时长。尽管推理延迟仍是开放挑战,但本工作为安全关键应用中的语言模型认证部署提供了可能。
原文摘要 · Abstract (English)
Understanding the reliability of natural language generation is critical for deploying foundation models in security-sensitive domains. While certified poisoning defenses provide provable robustness bounds for classification tasks, they are fundamentally ill-equipped for autoregressive generation: they cannot handle sequential predictions or the exponentially large output space of language models. To establish a framework for certified natural language generation, we formalize two security properties: stability (robustness to any change in generation) and validity (robustness to targeted, harmful changes in generation). We introduce Targeted Partition Aggregation (TPA), the first algorithm to certify validity/targeted attacks by computing the minimum poisoning budget needed to induce a specific harmful class, token, or phrase. Further, we extend TPA to provide tighter guarantees for multi-turn generations using mixed integer linear programming (MILP). Empirically, we demonstrate TPA's effectiveness across diverse settings including: certifying validity of agent tool-calling when adversaries modify up to 0.5% of the dataset and certifying 8-token stability horizons in preference-based alignment. Though inference-time latency remains an open challenge, our contributions enable certified deployment of language models in security-critical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。