用程序化剪枝让网页智能体高效定位操作目标
Prune4Web: DOM Tree Pruning Programming for Web Agent
- 让大模型生成可执行的Python脚本,动态筛选DOM节点
- 候选元素减少25至50倍,定位准确率从46.8%升至88.28%
- 适合做网页自动化、智能代理开发的研究者和工程师
网页自动化依赖智能体模拟人类操作网页界面来完成高阶任务。尽管基于大语言模型(LLM)的网页智能体已具备一定能力,但在复杂真实网页上高效导航仍面临挑战,主要源于文档对象模型(DOM)结构过大,通常达10,000至100,000个词元。现有方法多采用粗暴的DOM截断,易丢失关键信息,或使用低效启发式规则与独立排序模型,难以在精度与可扩展性间取得平衡。为此,我们提出Prune4Web,一种将DOM处理从资源密集型的LLM读取转向高效编程剪枝的新范式。核心是DOM树剪枝编程,即由大模型生成可执行的Python评分脚本,基于分解子任务的语义线索动态过滤DOM元素。该机制避免了大模型直接处理海量原始DOM,转而由轻量、可解释的程序负责遍历与打分。此方法使候选元素减少25至50倍,显著提升定位精度并缓解注意力稀释问题。此外,我们设计了专用数据标注流程与两轮对话训练策略,联合优化规划器、程序化过滤器与定位器。大量实验表明其达到领先性能:在低层级定位任务中,准确率从46.8%大幅提升至88.28%,验证了其在真实网页自动化中的有效性。
原文摘要 · Abstract (English)
Web automation employs intelligent agents to execute high-level tasks by mimicking human interactions with web interfaces. Despite the capabilities of recent Large Language Model (LLM)-based web agents, navigating complex, real-world webpages efficiently remains a significant hurdle due to the prohibitively large size of Document Object Model (DOM) structures, often ranging from 10,000 to 100,000 tokens. Existing strategies typically rely on crude DOM truncation -- risking the loss of critical information -- or employ inefficient heuristics and separate ranking models, failing to achieve an optimal balance between precision and scalability. To address these challenges, we introduce Prune4Web, a novel paradigm that shifts DOM processing from resource-intensive LLM reading to efficient programmatic pruning. Central to our approach is DOM Tree Pruning Programming, where an LLM generates executable Python scoring scripts to dynamically filter DOM elements based on semantic cues from decomposed sub-tasks. This mechanism eliminates the need for LLMs to ingest raw, massive DOMs, instead delegating traversal and scoring to lightweight, interpretable programs. This methodology achieves a 25x to 50x reduction in candidate elements for grounding, thereby facilitating precise action localization while mitigating attention dilution. Furthermore, we propose a specialized data annotation pipeline and a two-turn dialogue training strategy that jointly optimizes the Planner, Programmatic Filter, and Grounder within a unified framework. Extensive experiments demonstrate state-of-the-art performance. Notably, on our low-level grounding task, Prune4Web dramatically improves accuracy from 46.8% to 88.28%, underscoring its efficacy in real-world web automation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。