发现终端AI代理的拒绝列表普遍存在漏洞，易被绕过。
One Goal, Many Commands: Characterizing Denylist Fragility in AI Agents
- 用大模型生成潜在绕过命令，通过沙箱验证并迭代修复。
- 1709个真实拒绝列表中69%至98.6%存在可被绕过的漏洞。
- 揭示了拒绝列表脆弱性的根源，适合安全研究者与开发者参考。
AI代理的应用正在快速增长。终端AI代理（即在终端环境中运行的代理）是广泛应用的一类，它们依赖于shell命令执行与主机系统交互。为降低命令执行带来的安全风险，这类代理采用三列表命令过滤机制，其中拒绝列表（denylist）起关键作用。然而现代操作系统通常包含大量功能复杂的内置命令，我们观察到，即使是由Claude Code精心维护的内置拒绝列表，也可能遗漏能绕过其限制的命令，导致其无法阻止本应被拦截的操作。本文首次系统性地刻画了终端AI代理中命令拒绝列表的脆弱性问题。论文提出一种基于大模型的检测流水线ShellSieve，通过大模型生成可能的绕过方式，并借助沙箱验证器反馈进行迭代修复。我们在GitHub上收集的1,709个真实世界命令拒绝列表（共含13,332条规则）上评估了该方法。结果显示，69.0%–98.6%的拒绝列表存在脆弱性，且这种现象在不同项目和代理间持续出现，同时验证了若干潜在成因的合理性。本研究的工具与发现有望推动未来关于AI代理命令拒绝列表的研究与实践。
原文摘要 · Abstract (English)
The adoption of AI agents is increasing rapidly. Terminal AI agents, i.e., AI agents that run in terminal environments, are a widely used type of AI agents. Terminal AI agents rely heavily on shell command execution to interact with the host systems. They adopt a three-list command-gating mechanism to mitigate security risks introduced by command execution, with denylists serving as the load-bearing component. However, modern operating systems often ship a large, ever-expanding set of shell commands with complex functionalities. Our observation is that even a built-in denylist of Claude Code, well-maintained by its developers, can overlook bypass commands that invalidate its effectiveness. Such negligence leads to fragile command denylists that cannot even block operations that practitioners expect them to block. This paper presents the first systematic characterization of command denylist fragility in terminal AI agents. The paper formalizes the command denylist fragility problem and proposes an LLM-driven pipeline, ShellSieve, to detect such fragility. It prompts the LLM to propose possible bypasses and iteratively repairs them using feedback from a validator that executes them in a sandbox. In the evaluation, we applied ShellSieve to 1,709 real-world command denylists (containing 13,332 denylist rules) collected from GitHub. The evaluation shows several key findings, including that 69.0--98.6% of the denylists are fragile, that this fragility occurs consistently across projects and agents, and the validity of several possible root causes for this fragility. Our pipeline and findings will hopefully facilitate future research and practice regarding the command denylists used by AI agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。