用自然语言控制网页操作,让自动化更智能高效。
Steward: Natural Language Web Automation
- 通过自然语言指令驱动浏览器动作,自动规划执行流程。
- 单次操作耗时8.52至10.14秒,成本仅0.028美元,缓存后降至4.8秒。
- 适合研究推荐算法或需要大规模网页交互的开发者和研究人员。
大语言模型(LLMs)在构建人工智能助手方面展现出卓越能力,但在网页导航与界面元素交互方面的应用仍较有限。我们提出Steward,一种基于LLM的新型网页自动化工具,旨在提供低成本、可扩展、端到端的网页交互自动化解决方案。传统框架如Selenium、Puppeteer和Playwright在处理大规模或动态网页任务(如研究YouTube和Twitter的推荐算法)时存在可扩展性差的问题,需手动编写交互代码,限制了其适用性。Steward通过融合LLM能力与浏览器自动化,实现自然语言驱动的网站交互。它接收自然语言指令,动态规划并执行一系列动作,循环直至完成,为开发者和研究人员提供实用工具。系统在真实网站上运行,任务成功率达40%,单次操作耗时8.52至10.14秒,成本0.028美元,使用缓存机制后可降低至4.8秒和0.022美元。文中还讨论了状态表示、动作序列选择、系统响应性、任务完成检测及缓存实现等关键挑战。
原文摘要 · Abstract (English)
Recently, large language models (LLMs) have demonstrated exceptional capabilities in serving as the foundation for AI assistants. One emerging application of LLMs, navigating through websites and interacting with UI elements across various web pages, remains somewhat underexplored. We introduce Steward, a novel LLM-powered web automation tool designed to serve as a cost-effective, scalable, end-to-end solution for automating web interactions. Traditional browser automation frameworks like Selenium, Puppeteer, and Playwright are not scalable for extensive web interaction tasks, such as studying recommendation algorithms on platforms like YouTube and Twitter. These frameworks require manual coding of interactions, limiting their utility in large-scale or dynamic contexts. Steward addresses these limitations by integrating LLM capabilities with browser automation, allowing for natural language-driven interaction with websites. Steward operates by receiving natural language instructions and reactively planning and executing a sequence of actions on websites, looping until completion, making it a practical tool for developers and researchers to use. It achieves high efficiency, completing actions in 8.52 to 10.14 seconds at a cost of $0.028 per action or an average of $0.18 per task, which is further reduced to 4.8 seconds and $0.022 through a caching mechanism. It runs tasks on real websites with a 40% completion success rate. We discuss various design and implementation challenges, including state representation, action sequence selection, system responsiveness, detecting task completion, and caching implementation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。