用真实网页结构生成测试脚本，避免大模型胡编乱造
Autonomous QA Agent: A Retrieval-Augmented Framework for Reliable Selenium Script Generation
- 结合项目文档和网页结构检索，生成更可靠的自动化测试脚本
- 在20个电商场景中实现100%语法正确，90%执行成功
- 适合需要高可靠性自动化测试的开发团队
软件测试在开发流程中至关重要，但将需求转化为可执行测试脚本仍依赖人工且易出错。尽管大语言模型（LLM）能生成代码，却常虚构不存在的UI元素。我们提出自主QA代理（Autonomous QA Agent），一种基于检索增强生成（RAG）的系统，通过项目文档和实际HTML结构来约束Selenium脚本生成。系统将多种格式（Markdown、PDF、HTML）导入向量数据库，生成前检索相关上下文。在20个电商测试场景中的评估显示，该RAG方法达到100%（20/20）语法有效性，以及90%（18/20，95%置信区间：[85%, 95%]，p < 0.001）执行成功率，远超标准LLM生成的30%。尽管评估仅限单一领域，该方法显著降低幻觉问题，证明将生成过程锚定在真实DOM结构对自动化UI测试具有重要潜力。
原文摘要 · Abstract (English)
Software testing is critical in the software development lifecycle, yet translating requirements into executable test scripts remains manual and error-prone. While Large Language Models (LLMs) can generate code, they often hallucinate non-existent UI elements. We present the Autonomous QA Agent, a Retrieval-Augmented Generation (RAG) system that grounds Selenium script generation in project-specific documentation and HTML structure. By ingesting diverse formats (Markdown, PDF, HTML) into a vector database, our system retrieves relevant context before generation. Evaluation on 20 e-commerce test scenarios shows our RAG approach achieves 100% (20/20) syntax validity and 90% (18/20, 95% CI: [85%, 95%], p < 0.001) execution success, compared to 30% for standard LLM generation. While our evaluation is limited to a single domain, our method significantly reduces hallucinations by grounding generation in actual DOM structure, demonstrating RAG's potential for automated UI testing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。