测试编码代理在真实操作中是否偷偷执行危险行为。
Execution-Grounded Security Testing for Coding Agents in Software Engineering Pipelines
- 通过观察沙箱中的工具调用、运行时痕迹和文件变化来检测代理实际行为。
- 在代码载体上实现73.61%的危险操作触发率，文本载体达53.93%。
- 适合关注AI编程工具安全性的工程师与安全研究人员。
编码代理正越来越多地融入系统运维流程，其工具使用可直接修改项目文件、运行环境及底层系统。例如，若代理在启动或配置脚本中插入钩子，该更改可能持久存在并被后续触发，滥用授权权限对系统进行篡改。因此，安全测试必须关注代理的实际执行行为，而不仅是其输出内容。本文提出一种基于执行证据的红队测试框架，利用可观察的沙箱数据（如工具调用、运行时轨迹、文件系统差异）探测执行层的安全边界。框架将潜在危险操作嵌入常规软件工程任务（如单元测试、回归测试、崩溃复现与验证），并通过执行断言引导探针优化。在多个代理框架与模型架构下，经重构的任务工作负载使验证到的危险执行率显著提升：代码载体达73.61%，文本载体达53.93%。结果表明，一旦危险意图隐藏于合理工程任务中，编码代理仍可能在伪装下执行恶意操作。这说明系统级编码代理仍需更严格的安全部署与测试机制。
原文摘要 · Abstract (English)
Coding agents are increasingly integrated into system operations, where their tool use can directly modify project artifacts, execution environments, and the underlying system. For example, if a coding agent inserts a hook into a system startup or configuration script, that change can persist after the interaction, be triggered later, and abuse delegated user or system privileges to modify the system. This makes security testing a system problem: the key question is not only what the agent says, but what it actually does to the surrounding environment. We present an execution-grounded red-team testing framework for probing this execution-layer security boundary using observable sandbox evidence, including tool invocations, runtime traces, and file-system diffs. Our framework embeds target unsafe operations into routine software engineering workloads, including unit testing, regression testing, crash reproduction, and validation, and uses an execution oracle to guide refinement when an initial probe is rejected or fails. Across multiple agent frameworks and model backbones, our red-team workload reformulation substantially increases verified unsafe execution, reaching 73.61% on code carriers and 53.93% on text carriers. These results show that coding agents in system operations remain insecure under task disguise: once risky intent is hidden inside plausible engineering tasks, the agent can be induced to carry out unsafe actions on the surrounding system. More broadly, coding agents in system operations still demand stronger security testing and safeguards.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。