arXiv:2506.07524cs.SEcs.AI2025-06NeurIPS被引 8

用真实工具文档生成测试用例,发现大模型代理理解用户意图的漏洞。

TAI3: Testing Agent Integrity in Interpreting User Intent

  • 基于工具文档生成任务并针对性变异,暴露意图误判问题。
  • 在80个API上测试,错误发现率显著高于基线,查询效率更高。
  • 适合评估和改进依赖自然语言指令的AI代理系统。

大型语言模型代理通过自然语言指令调用API来自动化现实任务,但常因误解用户意图而偏离目标,尤其在外部工具包演进时更为明显。传统软件测试依赖结构化输入,难以应对自然语言的模糊性。本文提出TAI3,一个以API为中心的压力测试框架,可系统性地发现LLM代理中的意图完整性漏洞。不同于固定基准或对抗输入,TAI3基于工具包文档生成真实任务,并应用针对性变异以暴露细微错误,同时保持用户意图。为指导测试,提出语义划分,根据API参数及其等价类将自然语言任务分类。每个类别中,种子任务经变异并由轻量级预测器按触发错误概率排序。为提升效率,TAI3采用类型感知策略记忆,复用并适配过往有效变异模式。在80个工具包API上的实验表明,TAI3能有效发现意图完整性漏洞,在错误暴露率与查询效率上均显著优于基线。此外,TAI3可泛化至更强目标模型,使用较小语言模型生成测试用例,且能适应跨领域的演化API。

原文摘要 · Abstract (English)

LLM agents are increasingly deployed to automate real-world tasks by invoking APIs through natural language instructions. While powerful, they often suffer from misinterpretation of user intent, leading to the agent's actions that diverge from the user's intended goal, especially as external toolkits evolve. Traditional software testing assumes structured inputs and thus falls short in handling the ambiguity of natural language. We introduce TAI3, an API-centric stress testing framework that systematically uncovers intent integrity violations in LLM agents. Unlike prior work focused on fixed benchmarks or adversarial inputs, TAI3 generates realistic tasks based on toolkits' documentation and applies targeted mutations to expose subtle agent errors while preserving user intent. To guide testing, we propose semantic partitioning, which organizes natural language tasks into meaningful categories based on toolkit API parameters and their equivalence classes. Within each partition, seed tasks are mutated and ranked by a lightweight predictor that estimates the likelihood of triggering agent errors. To enhance efficiency, TAI3 maintains a datatype-aware strategy memory that retrieves and adapts effective mutation patterns from past cases. Experiments on 80 toolkit APIs demonstrate that TAI3 effectively uncovers intent integrity violations, significantly outperforming baselines in both error-exposing rate and query efficiency. Moreover, TAI3 generalizes well to stronger target models using smaller LLMs for test generation, and adapts to evolving APIs across domains.

Agent测试意图理解API安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。