arXiv:2603.26221cs.CRcs.AI2026-03被引 4

开源智能体系统安全风险难控,需建立可治理的防护体系。

Clawed and Dangerous: Can We Trust Open Agentic Systems?

  • 从软件工程视角构建六维分析框架,系统梳理智能体安全问题。
  • 发现现有研究攻击和评测较成熟,但部署控制与能力回收仍薄弱。
  • 提出安全构建指南与评估工具,适合平台开发者与安全审计者参考。

开源智能体系统结合大模型规划、外部能力、持久记忆和特权执行,广泛应用于代码助手、浏览器协作者和企业自动化。开放性使系统行为具有高度不确定性:计划动态生成,决策受不可信自然语言输入影响,执行环境模糊,行动权限由用户授予。传统软件的确定性控制流在此失效,核心挑战不再是抵御单一攻击,而是如何在持续不确定性下实现对智能体行为的有效治理。本文通过软件工程视角系统化该领域,提出六维分析分类法,并整合50篇相关论文(涵盖攻击、评测、防御、审计及工程基础)。基于此,提出安全构建准则与平台安全评估量表。研究发现,攻击表征与评测体系已相对成熟,但在部署管控、运行治理、持久记忆完整性和能力撤销方面仍存在明显短板。这些差距明确了构建可治理、可审计、抗妥协的智能体生态系统的具体工程方向。

原文摘要 · Abstract (English)

Open agentic systems combine LLM-based planning with external capabilities, persistent memory, and privileged execution. They are used in coding assistants, browser copilots, and enterprise automation. OpenClaw is a visible instance of this broader class. Without much attention yet, their security challenge is fundamentally different from that of traditional software that relies on predictable execution and well-defined control flow. In open agentic systems, everything is ''probabilistic'': plans are generated at runtime, key decisions may be shaped by untrusted natural-language inputs and tool outputs, execution unfolds in uncertain environments, and actions are taken under authority delegated by human users. The central challenge is therefore not merely robustness against individual attacks, but the governance of agentic behavior under persistent uncertainty. This paper systematizes the area through a software engineering lens. We introduce a six-dimensional analytical taxonomy and synthesize 50 papers spanning attacks, benchmarks, defenses, audits, and adjacent engineering foundations. From this synthesis, we derive a reference doctrine for secure-by-construction agent platforms, together with an evaluation scorecard for assessing platform security posture. Our review shows that the literature is relatively mature in attack characterization and benchmark construction, but remains weak in deployment controls, operational governance, persistent-memory integrity, and capability revocation. These gaps define a concrete engineering agenda for building agent ecosystems that are governable, auditable, and resilient under compromise.

智能体安全开源系统治理框架攻击评测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。