发现大模型工具系统中存在隐性持久语义实体，可能长期潜伏并传播污染。
Persistent Semantic Entities in Tool-Augmented LLM Systems
- 定义了'持久语义实体'，通过命名绑定、事件触发和跨边界传播实现
- 偏好与指令污染在所有模型中持续存在，最长10轮不衰减
- 标准监控难捕捉，自验证可减少36.5%污染，适合部署安全评估
工具增强的大型语言模型代理可能在会话间隐含持久状态，通过事件激活并跨代理边界传播，常规调试难以察觉。本文将其形式化为持久语义实体（PSEs）：由命名绑定、事件触发和跨边界传播构成，并在11个模型家族的24个模型（参数量1.5B–1T）上进行评估。所有测试模型均易受攻击（20%-100%在20模型敏感性面板中），命名绑定是必要且主导机制：无此机制时污染为0%。持久性取决于污染类型而非模型规模或部署方式：偏好污染在所有探测模型中均未衰减（t=10时仍100%），指令污染一旦采用即持续存在，人格注入部分衰减（90%→10%），而事实注入则因模型而异——在Llama-3.1-8B和GPT-4o-mini中自我修正，但在两个Qwen2.5-coder变体中保持顶峰水平，故不能泛化认为其具备自修正能力。偏好与指令结果在控制环境下跨提供商一致。上下文隔离的自我验证可实现20%-79%（中位数36.5%）的污染降低，无需参考答案；关键词检测则产生系统性误报。污染在四阶段代理流水线中放大1.9倍（40%→75%）。偏好与指令污染——持久、无自纠、难被标准监测捕获——构成了部署型代理系统尤其严峻的安全威胁。
原文摘要 · Abstract (English)
Tool-augmented LLM agents can harbor implicit state that persists across sessions, activates through events, and propagates across agent boundaries---largely invisible to standard debugging. We formalize this as Persistent Semantic Entities (PSEs): constructs defined by name binding, event triggering, and cross-boundary propagation, and evaluate them across 24 models from 11 families (1.5B--1T parameters). First, every tested model is susceptible (20--100% on the 20-model susceptibility panel), with name binding as the necessary and dominant mechanism: without it, contamination is 0%. Second, persistence depends on contamination type rather than scale or deployment: preference contamination persists undecayed on every model probed (100% at t=10) and instruction contamination persists wherever adopted, persona-style injection decays partially (90%$\to$10%), while factual injection is model-dependent---self-corrected on Llama-3.1-8B and GPT-4o-mini but held at ceiling on both Qwen2.5-coder variants, so we do not claim it self-corrects in general. The preference and instruction results hold across providers in our controlled setting. Third, context-isolated self-verification achieves 20--79% reduction (median 36.5%) without oracle references while keyword-based detection produces systematic false positives, and contamination compounds 1.9$\times$ along a four-stage agent pipeline (40%$\to$75%). Preference and instruction contamination---persistent, lacking self-correction, and poorly captured by standard monitoring---represent a particularly concerning attack surface for deployed agent systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。