用自然语言自动生成精准XPath，提升爬虫开发效率
XPath Agent: An Efficient XPath Programming Agent Based on LLM for Web Crawler
- 单次自然语言指令生成XPath，无需手动编写
- 同等性能下减少70%以上令牌消耗，提速显著
- 两阶段流程易集成，适合自动化测试与爬虫项目
我们提出XPath Agent，一个专为网络爬虫和网页GUI测试设计的生产级XPath编程代理。其核心能力是仅通过一次自然语言查询，即可从一组采样网页中自动构建出准确的XPath表达式。为验证有效性，我们在多项网络爬取任务上对比了当前最先进的XPath编程代理。结果表明，XPath Agent在性能指标相当的前提下，大幅降低令牌使用量并提升时钟时间效率。精心设计的两阶段流水线可无缝融入现有爬虫或测试工作流，显著节省手动编写XPath所需的时间与精力。代码已开源：https://github.com/eavae/feilian。
原文摘要 · Abstract (English)
We present XPath Agent, a production-ready XPath programming agent specifically designed for web crawling and web GUI testing. A key feature of XPath Agent is its ability to automatically generate XPath queries from a set of sampled web pages using a single natural language query. To demonstrate its effectiveness, we benchmark XPath Agent against a state-of-the-art XPath programming agent across a range of web crawling tasks. Our results show that XPath Agent achieves comparable performance metrics while significantly reducing token usage and improving clock-time efficiency. The well-designed two-stage pipeline allows for seamless integration into existing web crawling or web GUI testing workflows, thereby saving time and effort in manual XPath query development. The source code for XPath Agent is available at https://github.com/eavae/feilian.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。