提出防御内存投毒攻击的新方法,能精准识别恶意篡改且具理论保证。
MEMSAD: Gradient-Coupled Anomaly Detection for Memory Poisoning in Retrieval-Augmented Agents
- 基于梯度耦合原理,设计可证明安全的异常检测机制。
- 在多种攻击下实现100%检出率、0%误报率,性能显著优于现有方法。
- 适合关注大模型安全性与可信推理的科研人员和工程师。
持久化外部记忆使大语言模型代理能在会话间保持上下文,但其安全特性尚未得到形式化分析。本文将检索增强型代理的内存投毒攻击建模为一种统一框架下的斯塔克尔伯格博弈,涵盖三类逐步强化访问假设的攻击。修正了Chen等(2024)在触发查询设定中的评估不一致问题,发现真实评估使攻击成功率提升至4倍(ASR-R从0.25升至1.00)。核心贡献是MEMSAD(语义异常检测)——一种基于梯度耦合定理的校准防御:在编码器规则性条件下,异常得分梯度与检索目标梯度完全一致,因此任何降低检测风险的连续扰动必然损害检索排名。该耦合关系带来可证明的检测半径保障,确保分类正确性不受对抗策略影响。通过勒卡姆方法证明极小最优性,表明任意阈值检测器需Ω(1/ρ²)校准样本,而MEMSAD达到此量级(仅差log(1/δ)因子)。进一步推导滚动校准的在线后悔上界为O(σ^{2/3}Δ^{1/3}),并形式化揭示离散同义词不变性漏洞,定义连续空间防御的极限边界。在3×5攻击-防御矩阵上的实验(n=1,000,Bootstrap置信区间、邦弗伦尼校正假设检验、Clopper-Pearson验证)显示:组合防御实现100%真正例率、0%假正例率,而同义词替换攻击在Δ ASR-R≈0时逃逸检测,暴露当前嵌入式防御无法弥补的差距。
原文摘要 · Abstract (English)
Persistent external memory enables LLM agents to maintain context across sessions, yet its security properties remain formally uncharacterized. We formalize memory poisoning attacks on retrieval-augmented agents as a Stackelberg game with a unified evaluation framework spanning three attack classes with escalating access assumptions. Correcting an evaluation protocol inconsistency in the triggered-query specification of Chen et al. (2024), we show faithful evaluation increases measured attack success by $4\times$ (ASR-R: $0.25 \to 1.00$). Our primary contribution is MEMSAD (Semantic Anomaly Detection), a calibration-based defense grounded in a gradient coupling theorem: under encoder regularity, the anomaly score gradient and the retrieval objective gradient are provably identical, so any continuous perturbation that reduces detection risk necessarily degrades retrieval rank. This coupling yields a certified detection radius guaranteeing correct classification regardless of adversary strategy. We prove minimax optimality via Le Cam's method, showing any threshold detector requires $Ω(1/ρ^2)$ calibration samples and MEMSAD achieves this up to $\log(1/δ)$ factors. We further derive online regret bounds for rolling calibration at rate $O(σ^{2/3}Δ^{1/3})$, and formally characterize a discrete synonym-invariance loophole that marks the boundary of what continuous-space defenses can guarantee. Experiments on a $3 \times 5$ attack-defense matrix with bootstrap confidence intervals, Bonferroni-corrected hypothesis tests, and Clopper-Pearson validation ($n=1{,}000$) confirm: composite defenses achieve TPR $= 1.00$, FPR $= 0.00$ across all attacks, while synonym substitution evades detection at $Δ$ ASR-R $\approx 0$, exposing a gap existing embedding-based defenses cannot close.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。