arXiv:2608.02518cs.AIcs.CY2026-08

用跨会话能力累积检测AI滥用,破解攻击者分步隐藏的漏洞。

Magnet: Detecting Cross-Session AI Misuse Through Capability Accumulation

论文配图:Magnet: Detecting Cross-Session AI Misuse Through Capability Accumulation
图 1 · 摘自论文原文
  • 通过跨会话分解目标,将有害行为拆成无害小任务
  • 相比单会话攻击,可触发更强危害性能力组合
  • 提出Magnet系统,聚合用户级证据捕捉隐蔽威胁

最强大的AI部署并非单一模型,而是由多个专业代理协同工作的集成系统。这种架构虽带来强大能力,却也引入了现有监控、检测与缓解框架无法应对的风险。当前主流的AI滥用检测研究集中于单轮或多轮(单会话)威胁模型,留下关键空白:攻击者可将有害目标分解为看似无害的单元,并在隔离的代理会话中逐一执行。代理在对话间无状态,但攻击者有。这种不对称性使跨会话路径能有效规避检测。本文贡献有二:首先,揭示跨会话目标分解作为逃避技术的可行性,表明其可能引发比等效单会话或多轮攻击更强的危害能力;所谓‘能力’指在某一步骤产生的成果(模型响应与工具调用结果),可与其他阶段积累的能力组合成有害整体。其次,提出Magnet——一种高效稳健的检测方法,通过建模随时间与跨会话累积的相关能力,在更高层级(如用户ID)进行聚合,而非依赖单会话状态。核心挑战在于构建Magnet推理所需的证据包。可疑痕迹如同散落在大量良性会话中的针,单独看均无害,仅在聚合后才显危险。Magnet不逐会话排查(即‘逐草寻针’),而如其名所示:将相关‘针’从‘干草堆’中吸引出来,跨会话、跨时间汇聚成紧凑证据包,供检测器处理。

原文摘要 · Abstract (English)

The most capable AI deployments are not single models but ensembles of specialized agents that delegate and act in coordination. This architecture unlocks powerful new capabilities, and it also introduces risks that existing frameworks for monitoring, detection, and mitigation were not designed to address. Most state-of-the-art AI abuse detection literature focuses on single-turn or multi-turn (single-session) threat models. This leaves a critical gap: an attacker can decompose a harmful goal into innocuous-looking units and execute each in isolated agentic sessions. The agent is stateless between conversations, but the attacker is not. This asymmetry allows for cross-session trajectories that are effective at evading detection. Our contributions are twofold. First, we demonstrate cross-session goal decomposition as an evasion technique, showing it may elicit more harmful capability than equivalent single-session or multi-turn attacks. By capability we mean an artifact produced at one step of an objective, evidenced by what an interaction produced (model responses and tool-call results), and composable with capabilities accrued elsewhere into a harmful whole. Second, we propose Magnet: an efficient and robust detection approach that models relevant capabilities accrued over time and across agentic conversations, aggregated at a higher-level correlator (in this case, a user ID) rather than per-conversation state. The main challenge is assembling the evidence bundle Magnet reasons over. The incriminating artifacts may be needles scattered through a haystack of benign sessions that are individually harmless, dangerous only once collected. Rather than searching the haystack straw-by-straw (i.e. per-session inspection), Magnet does what its name implies: it attracts the relevant needles out of the hay, across sessions and across time, into a compact evidence bundle a detector can act on.

AI安全跨会话检测机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。