对比四种网页交互方式，发现新接口显著提升AI代理效率与准确率。
MCP vs RAG vs NLWeb vs HTML: A Comparison of the Effectiveness and Efficiency of Different Agent Interfaces to the Web (Technical Report)
- 用统一测试环境比较HTML、RAG、MCP和NLWeb四种网页交互方式。
- RAG等新接口使任务准确率提升至0.87，耗时降至62秒以内。
- 适合关注AI代理性能优化的研究者与开发者参考。
大型语言模型代理正被用于自动化商品搜索、价格比对和结账等网络任务。现有研究探索了多种与网站交互的接口，包括传统HTML浏览、基于预爬取内容的检索增强生成（RAG）、通过模型上下文协议（MCP）的API通信，以及自然语言查询的NLWeb接口。然而，此前尚无研究在相同条件下对这四种架构进行系统性对比。为此，我们构建了一个包含四个模拟电商站点的测试平台，每个站点提供HTML、MCP和NLWeb接口。针对每种接口，我们开发了执行相同任务的专用代理，涵盖从简单商品搜索到复杂互补或替代品查询及结账流程。使用GPT 4.1、GPT 5、GPT 5 mini和Claude Sonnet 4作为底层大模型进行评估。结果表明，RAG、MCP和NLWeb代理在有效性和效率上均优于HTML：平均F1得分从HTML的0.67提升至0.75–0.77；每任务令牌消耗从约241k降至47k–140k；任务运行时间从291秒降至50–62秒。最佳配置为使用GPT 5的RAG方案，达F1 0.87，完成率0.79。综合成本考虑，GPT 5 mini的RAG方案在性能与费用间取得良好平衡。实验表明，交互接口的选择对基于大模型的网页代理表现有显著影响。
原文摘要 · Abstract (English)
Large language model agents are increasingly used to automate web tasks such as product search, offer comparison, and checkout. Current research explores different interfaces through which these agents interact with websites, including traditional HTML browsing, retrieval-augmented generation (RAG) over pre-crawled content, communication via Web APIs using the Model Context Protocol (MCP), and natural-language querying through the NLWeb interface. However, no prior work has compared these four architectures within a single controlled environment using identical tasks. To address this gap, we introduce a testbed consisting of four simulated e-shops, each offering its products via HTML, MCP, and NLWeb interfaces. For each interface (HTML, RAG, MCP, and NLWeb) we develop specialized agents that perform the same sets of tasks, ranging from simple product searches and price comparisons to complex queries for complementary or substitute products and checkout processes. We evaluate the agents using GPT 4.1, GPT 5, GPT 5 mini, and Claude Sonnet 4 as underlying LLM. Our evaluation shows that the RAG, MCP and NLWeb agents outperform HTML on both effectiveness and efficiency. Averaged over all tasks, F1 rises from 0.67 for HTML to between 0.75 and 0.77 for the other agents. Token usage falls from about 241k for HTML to between 47k and 140k per task. The runtime per task drops from 291 seconds to between 50 and 62 seconds. The best overall configuration is RAG with GPT 5 achieving an F1 score of 0.87 and a completion rate of 0.79. Also taking cost into consideration, RAG with GPT 5 mini offers a good compromise between API usage fees and performance. Our experiments show the choice of the interaction interface has a substantial impact on both the effectiveness and efficiency of LLM-based web agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。