arXiv:2608.27990cs.CRcs.AI2026-08

让AI自动生成防御新攻击的方案,能自动发现并应对未知注入威胁。

CAITLYN: Can LLM Agents Autonomously Synthesize Defenses against Emerging Injection Attacks?

论文配图:CAITLYN: Can LLM Agents Autonomously Synthesize Defenses against Emerging Injection Attacks?
图 1 · 摘自论文原文
  • 设计双系统架构:规则+LLM快速检测,持续监控并自主生成新防御。
  • 在新攻击基准上,系统二使攻击成功率大幅下降,而传统方法失效。
  • 适合研究者和开发者,解决现有防御难适应新攻击的问题。

大型语言模型(LLM)代理面临的提示注入攻击,旨在将恶意指令或内容引入代理检索的外部文本源,迫使底层LLM执行超出其安全范围的有害操作。尽管现有防御对已知攻击有效,但在LLM代理环境中部署仍面临攻击变种和新兴威胁的挑战。此外,现有方案通常陷入运行效率、上下文精度与适应性之间的固有三难困境。为此,我们提出连续代理注入威胁防御机制——CAITLYN,一种代理无关的防御中间件。CAITLYN集成两个系统:系统I采用两级库实现即时防御,0级为基于规则的检测脚本,1级为优化后的LLM精准推理;系统II用于监控异常信号并尝试自主合成新防御。在标准基准上,CAITLYN以更低的令牌开销达到先进防御的检测性能。在新提出的交付感知基准Emerging上,静态基线与系统I配置仍易受攻击;而系统II自主生成经验证的防御能力,在三个不同代理环境中显著降低攻击成功率。

原文摘要 · Abstract (English)

Prompt injection attacks on Large Language Model (LLM) agents seek to introduce malicious instructions or content into external text sources retrieved by agents, forcing the underlying LLMs to execute harmful actions outside their benign scope. While current defenses effectively counter known injection attacks, deploying them in LLM agent environments remains challenging due to attack variants and emerging threats. Moreover, existing solutions typically suffer from an inherent trilemma, i.e., a constant trade-off among runtime efficiency, contextual precision, and adaptability. To bridge this gap, we propose Continuous Agents for Injection Threats via Lifelong Yielding Nexus (CAITLYN), an agent-agnostic defense middleware. CAITLYN integrates two systems. System I focuses on immediate defense against existing attacks using a two-tiered library: Tier-0 for rule-based detection scripts and Tier-1 for optimized LLM-based accurate inference. System II, in contrast, is deployed to monitor potential abnormal signals and attempt to synthesize new defenses. On standard benchmarks, CAITLYN matches the detection performance of state-of-the-art defenses at lower token overhead than LLM-as-a-judge baselines. On Emerging, our new delivery-aware benchmark featuring novel injection techniques, static baselines and the standalone System I configuration remain vulnerable. In contrast, System II autonomously synthesizes verified defense capabilities, substantially lowering the attack success rate across three diverse agent environments.

AI安全防御机制LLM代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。