通过推理时扩增计算资源，提升大模型抗提示注入攻击能力。
SecInfer: Preventing Prompt Injection via Inference-time Scaling
- 利用多样化系统提示生成多条推理路径，增强响应多样性。
- 在额外计算开销下，对齐任务目标的响应选择，显著降低攻击成功率。
- 适用于需高安全性的实际部署场景，尤其对抗自适应攻击有效。
提示注入攻击严重威胁大语言模型的安全性。现有基于微调的防御方法在面对强攻击时效果有限。本文提出SecInfer，一种基于推理时扩增（inference-time scaling）的新防御机制，通过增加推理阶段的计算资源来提升安全性。其核心包含两步：系统提示引导采样，利用不同系统提示探索多种推理路径以生成多个响应；目标任务引导聚合，从多个响应中选出最可能完成目标任务的那个。大量实验表明，借助推理时的额外计算，SecInfer能有效抵御已有及自适应提示注入攻击，性能优于当前最优防御方案和现有推理时扩增方法。
原文摘要 · Abstract (English)
Prompt injection attacks pose a pervasive threat to the security of Large Language Models (LLMs). State-of-the-art prevention-based defenses typically rely on fine-tuning an LLM to enhance its security, but they achieve limited effectiveness against strong attacks. In this work, we propose \emph{SecInfer}, a novel defense against prompt injection attacks built on \emph{inference-time scaling}, an emerging paradigm that boosts LLM capability by allocating more compute resources for reasoning during inference. SecInfer consists of two key steps: \emph{system-prompt-guided sampling}, which generates multiple responses for a given input by exploring diverse reasoning paths through a varied set of system prompts, and \emph{target-task-guided aggregation}, which selects the response most likely to accomplish the intended task. Extensive experiments show that, by leveraging additional compute at inference, SecInfer effectively mitigates both existing and adaptive prompt injection attacks, outperforming state-of-the-art defenses as well as existing inference-time scaling approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。