用测试驱动方式生成可落地的智能体，确保行为符合预期。
Test-Driven AI Agent Definition (TDAD): Compiling Tool-Using Agents from Behavioral Specifications
- 将行为规范转为可执行测试，通过双智能体迭代优化提示词。
- 92%成功率编译通过，隐藏测试通过率达97%，变异测试检测能力86%-100%。
- 适合需要高可靠性的工具使用型AI智能体开发团队使用。
我们提出测试驱动的AI智能体定义（TDAD），将智能体提示词视为编译产物：工程师提供行为规范，编码智能体将其转化为可执行测试，第二个编码智能体持续优化提示词直至测试通过。当前开发流程无法保障工具使用型大模型智能体的行为可测量合规性，微小提示修改可能导致无声退化，工具误用未被发现，政策违规仅在部署后暴露。为缓解规范博弈问题，TDAD引入三项机制：(1) 可见/隐藏测试划分，在编译阶段隐藏评估测试；(2) 编译后语义变异测试，由代理生成合理错误提示变体，测试套件衡量其检出能力；(3) 规范演进场景，量化需求变更下的回归安全性。我们在SpecSuite-Core上评估，该基准包含四个深度规范的智能体，涵盖政策合规、基于证据分析、运行手册遵循和确定性执行。24次独立实验中，TDAD实现92% v1编译成功率，平均隐藏测试通过率97%；演化后的规范编译成功率为58%，多数失败案例仅在1-2个隐藏测试中失败，变异测试得分86%-100%，v2隐藏通过率78%，回归安全评分97%。代码实现已开源，详见https://github.com/f-labs-io/tdad-paper-code。
原文摘要 · Abstract (English)
We present Test-Driven AI Agent Definition (TDAD), a methodology that treats agent prompts as compiled artifacts: engineers provide behavioral specifications, a coding agent converts them into executable tests, and a second coding agent iteratively refines the prompt until tests pass. Deploying tool-using LLM agents in production requires measurable behavioral compliance that current development practices cannot provide. Small prompt changes cause silent regressions, tool misuse goes undetected, and policy violations emerge only after deployment. To mitigate specification gaming, TDAD introduces three mechanisms: (1) visible/hidden test splits that withhold evaluation tests during compilation, (2) semantic mutation testing via a post-compilation agent that generates plausible faulty prompt variants, with the harness measuring whether the test suite detects them, and (3) spec evolution scenarios that quantify regression safety when requirements change. We evaluate TDAD on SpecSuite-Core, a benchmark of four deeply-specified agents spanning policy compliance, grounded analytics, runbook adherence, and deterministic enforcement. Across 24 independent trials, TDAD achieves 92% v1 compilation success with 97% mean hidden pass rate; evolved specifications compile at 58%, with most failed runs passing all visible tests except 1-2, and show 86-100% mutation scores, 78% v2 hidden pass rate, and 97% regression safety scores. The implementation is available as an open benchmark at https://github.com/f-labs-io/tdad-paper-code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。