arXiv:2604.21131cs.CRcs.AI2026-04被引 2

提出跨会话攻击检测新基准,解决AI代理因无记忆而漏检分段攻击的问题

Cross-Session Threats in AI Agents: Benchmark, Evaluation, and Algorithms

论文配图:Cross-Session Threats in AI Agents: Benchmark, Evaluation, and Algorithms
图 1 · 摘自论文原文
  • 构建包含26类攻击的CSTM-Bench数据集,区分攻击阶段与跨会话操作模式
  • 发现现有检测方法在跨会话场景下召回率下降近一半,暴露上下文窗口局限性
  • 设计带记忆的Coreset Memory Reader与新指标CSR_prefix,提升检测稳定性

AI代理的防护机制是无记忆的:每个消息独立判断,导致攻击者将单一攻击分散至数十个会话时,可避开所有会话级检测器,因只有整体才携带攻击载荷。本文提出三项贡献:(1)构建CSTM-Bench数据集,包含26类可执行攻击,按杀伤链阶段与跨会话操作(积累、组合、清洗、读者注入)分类,绑定七种身份锚点以真实标注违规行为,并提供良性纯净与良性困难对照样本;数据已发布于Hugging Face(intrinsec-ai/cstm-bench),含两种54场景划分:稀释(组合式)与跨会话(由闭环重写器生成的12个会话隔离不可见场景,弱化表面表述但保留跨会话痕迹)。(2)将跨会话检测建模为下游相关器LLM的信息瓶颈,发现会话级判断器与全日志串联相关器在从稀释到跨会话迁移时,攻击召回率均下降约一半,远低于任何前沿上下文窗口容量。测试范围:每片54场景,一种相关器家族(Anthropic Claude),无提示优化;释放此数据以激励更大规模多供应商数据集。(3)提出有界内存的Coreset Memory Reader,在K=50时唯一保持双片召回率;因排序器打乱破坏KV缓存前缀复用,故将$  ext{CSR} \_\text{prefix}$(有序前缀稳定性,无需LLM)提升为第一类指标,并融合检测形成$ \text{CSTM} = 0.7 F_1(\text{CSDA@action}, \text{precision}) + 0.3 \text{CSR}\_\text{prefix}$,在单个帕累托前沿上评估排序器的召回率与服务稳定性。

原文摘要 · Abstract (English)

AI-agent guardrails are memoryless: each message is judged in isolation, so an adversary who spreads a single attack across dozens of sessions slips past every session-bound detector because only the aggregate carries the payload. We make three contributions to cross-session threat detection. (1) Dataset. CSTM-Bench is 26 executable attack taxonomies classified by kill-chain stage and cross-session operation (accumulate, compose, launder, inject_on_reader), each bound to one of seven identity anchors that ground-truth "violation" as a policy predicate, plus matched Benign-pristine and Benign-hard confounders. Released on Hugging Face as intrinsec-ai/cstm-bench with two 54-scenario splits: dilution (compositional) and cross_session (12 isolation-invisible scenarios produced by a closed-loop rewriter that softens surface phrasing while preserving cross-session artefacts). (2) Measurement. Framing cross-session detection as an information bottleneck to a downstream correlator LLM, we find that a session-bound judge and a Full-Log Correlator concatenating every prompt into one long-context call both lose roughly half their attack recall moving from dilution to cross_session, well inside any frontier context window. Scope: 54 scenarios per shard, one correlator family (Anthropic Claude), no prompt optimisation; we release it to motivate larger, multi-provider datasets. (3) Algorithm and metric. A bounded-memory Coreset Memory Reader retaining highest-signal fragments at $K=50$ is the only reader whose recall survives both shards. Because ranker reshuffles break KV-cache prefix reuse, we promote $\mathrm{CSR\_prefix}$ (ordered prefix stability, LLM-free) to a first-class metric and fuse it with detection into $\mathrm{CSTM} = 0.7 F_1(\mathrm{CSDA@action}, \mathrm{precision}) + 0.3 \mathrm{CSR\_prefix}$, benchmarking rankers on a single Pareto of recall versus serving stability.

AI安全攻击检测跨会话评测基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。