arXiv:2511.00447cs.CRcs.AI2025-11被引 3

提出DRIP防御提示注入,精准清除数据中指令语义且不丢失信息

DRIP: Defending Prompt Injection via Token-wise Representation Editing and Residual Instruction Fusion

  • 通过轻量级嵌入编辑模块,剥离数据中类指令的语义
  • 在对抗攻击下将攻击成功率降低66%以上,同时保持模型原有性能
  • 适合需高安全性的LLM应用,如企业级AI系统

大型语言模型(LLMs)被广泛集成到IT基础设施中,根据预设指令处理用户数据。然而,传统LLMs仍易受提示注入攻击,恶意用户通过在数据中插入指令类标记来操控模型行为。现有防御方法虽尝试分离数据与指令语义,但仍面临(1)效用与安全难以平衡、(2)数据中类指令语义可能覆盖原始指令的问题。本文提出DRIP,通过(1)精准移除数据段中指令语义而保留其数据语义,(2)确保在强对抗内容下仍能维持原指令效果。DRIP引入数据清洗与训练范式,配备轻量级表示编辑模块,对数据段中类指令令牌的嵌入进行编辑,提升安全性而不损害效用;并通过添加最小残差模块,降低对抗数据覆盖原始指令的能力。我们在LLaMA 8B和Mistral 7B上评估DRIP,在SEP、AlpacaFarm和InjecAgent三个提示注入基准测试中,相比StruQ、SecAlign、ISE和PFT,DRIP使角色分离得分提升12-49%,在自适应攻击下攻击成功率降低超66%,且效用与未受保护模型相当,建立了提示注入鲁棒性新基准。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly integrated into IT infrastructures, where they process user data according to predefined instructions. However, conventional LLMs remain vulnerable to prompt injection, where malicious users inject directive tokens into the data to subvert model behavior. Existing defenses train LLMs to semantically separate data and instruction tokens, but still struggle to (1) balance utility and security and (2) prevent instruction-like semantics in the data from overriding the intended instructions. We propose DRIP, which (1) precisely removes instruction semantics from tokens in the data section while preserving their data semantics, and (2) robustly preserves the effect of the intended instruction even under strong adversarial content. To "de-instructionalize" data tokens, DRIP introduces a data curation and training paradigm with a lightweight representation-editing module that edits embeddings of instruction-like tokens in the data section, enhancing security without harming utility. To ensure non-overwritability of instructions, DRIP adds a minimal residual module that reduces the ability of adversarial data to overwrite the original instruction. We evaluate DRIP on LLaMA 8B and Mistral 7B against StruQ, SecAlign, ISE, and PFT on three prompt-injection benchmarks (SEP, AlpacaFarm, and InjecAgent). DRIP improves role-separation score by 12-49\%, reduces attack success rate by over 66\% under adaptive attacks, and matches the utility of the undefended model, establishing a new state of the art for prompt-injection robustness.

提示注入安全防御大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。