发现大模型代理内存中毒的轨迹特征，可高效检测攻击行为。
Forensic Trajectory Signatures for Agent Memory Poisoning Detection
- 利用记忆调用顺序的不变性构建检测规则
- 在多模型上达到0.99的检测准确率，跨模型泛化能力强
- 适合用于实时安全防护，需结合收件人信息降低误报
我们发现，在持续内存中毒攻击下，大型语言模型代理存在一种行为不变性，并刻画了其部署边界。在检索通过可观测的记忆工具调用路由的架构中，成功攻击必须在发送邮件前调用记忆召回，这一顺序由攻击的信息检索依赖强制形成。基于此不变性设计的简单规则实现AUC=0.9563；使用19个轨迹特征的随机森林进一步提升至AUC=0.9904（BCa 95%置信区间[0.987, 0.993]）。该签名在中毒但受保护的数据集上高度冗余：移除所有回忆相关特征后AUC不变。在9个模型（7B-120B）上的跨模型留出测试中，6/9分割的AUC达1.000，且该不变性可直接迁移至前沿模型（GPT-4.1、GPT-4o），无需重新训练。[v2] 预注册后续研究（N=4,360，13个模型）揭示关键部署边界：良性记忆驱动的邮件发送也会产生相同的‘先召回后发送’特征，导致条件为真时误报率达100%（无条件良性误报率：24.7%-52.6%，取决于召回协议）。该签名是攻击前提，而非恶意判定，独立阻断不可行，但结合收件人元数据可恢复区分能力。仅用前缀的变体实现AUC=0.934，支持实时筛查。
原文摘要 · Abstract (English)
We discover a behavioral invariant in LLM agents under persistent memory poisoning and characterize its deployment boundary. In architectures where retrieval is routed through observable memory-tool invocations, successful attacks require calling memory_recall_fact before email_send_email, a transition mechanistically forced by the attack's information-retrieval dependency. A simple rule exploiting this invariant achieves AUC = 0.9563; a Random Forest over 19 trajectory features refines it to AUC = 0.9904 (BCa 95% CI [0.987, 0.993]). The signature is overdetermined within the poisoned-but-defended evaluation set: removing all recall-related features leaves AUC unchanged. Cross-model hold-out on 9 models (7B-120B) confirms AUC = 1.000 on 6/9 splits, and the invariant transfers to frontier models (GPT-4.1, GPT-4o) without retraining. [v2] A preregistered follow-up (N=4,360, 13 models) reveals a critical deployment boundary: benign memory-grounded sends produce the same recall_before_send signature, yielding 100% false positives conditional on recall_before_send=1 (unconditional benign FPR: 24.7-52.6% depending on recall protocol). The signature is a valid attack precondition, not a maliciousness predicate; standalone blocking is not viable, but gating with recipient metadata restores separation. A prefix-only variant achieves AUC = 0.934, enabling real-time triage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。