arXiv:2504.19956cs.CRcs.AI2025-04被引 57

为自主生成式智能体构建全新安全威胁模型,防范其自主行为带来的新型风险。

Securing Agentic AI: A Comprehensive Threat Model and Mitigation Framework for Generative AI Agents

  • 提出针对智能体特性的五类威胁域,涵盖认知、记忆、执行等维度
  • 识别9种核心威胁,包括隐蔽目标错位和跨系统横向移动等现实风险
  • 提供ATFAA与SHIELD双框架,适配企业部署的智能体安全防护

随着生成式AI(GenAI)代理在企业中日益普及,其带来的安全挑战与传统系统截然不同。这些代理不仅具备大语言模型能力,还能推理、记忆并自主行动,常在极少人工干预下运行。本文针对此类代理特性,提出一个全面的威胁模型,聚焦其自主性、持续内存访问、复杂推理及工具集成所引发的新风险。研究识别出9种主要威胁,并将其归入五个关键领域:认知架构漏洞、时间持久性威胁、操作执行漏洞、信任边界越界以及治理绕过。这些威胁不仅是理论上的,更带来实际问题,如延迟可利用性、跨系统传播、横向移动及难以察觉的目标错位。为此,本文提出两个互补框架:ATFAA(面向自主智能体的高级威胁框架),用于组织代理相关风险;以及SHIELD,提出可落地的缓解策略,以降低企业暴露面。该研究虽基于现有大模型与AI安全工作,但重点在于揭示代理的独特性及其对安全的影响。最终主张,必须为生成式智能代理建立新的安全视角。若不适应其独特架构与行为更新威胁模型与防御体系,这一强大新工具可能演变为严重的企业风险。

原文摘要 · Abstract (English)

As generative AI (GenAI) agents become more common in enterprise settings, they introduce security challenges that differ significantly from those posed by traditional systems. These agents are not just LLMs; they reason, remember, and act, often with minimal human oversight. This paper introduces a comprehensive threat model tailored specifically for GenAI agents, focusing on how their autonomy, persistent memory access, complex reasoning, and tool integration create novel risks. This research work identifies 9 primary threats and organizes them across five key domains: cognitive architecture vulnerabilities, temporal persistence threats, operational execution vulnerabilities, trust boundary violations, and governance circumvention. These threats are not just theoretical they bring practical challenges such as delayed exploitability, cross-system propagation, cross system lateral movement, and subtle goal misalignments that are hard to detect with existing frameworks and standard approaches. To help address this, the research work present two complementary frameworks: ATFAA - Advanced Threat Framework for Autonomous AI Agents, which organizes agent-specific risks, and SHIELD, a framework proposing practical mitigation strategies designed to reduce enterprise exposure. While this work builds on existing work in LLM and AI security, the focus is squarely on what makes agents different and why those differences matter. Ultimately, this research argues that GenAI agents require a new lens for security. If we fail to adapt our threat models and defenses to account for their unique architecture and behavior, we risk turning a powerful new tool into a serious enterprise liability.

AI安全智能体威胁建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。