让AI助手更快更准地操作网页,减少87%的计算开销。
PAFFA: Premeditated Actions For Fast Agents
- 构建动作库,预计算通用浏览器交互模式,减少重复推理。
- 推理耗时降低87%,任务准确率提升至0.57(基线0.50)。
- 无需训练即可适应新网站,适合快速部署的智能代理场景。
现代AI助手在自然语言理解和工具使用方面取得显著进展,尤其在与网页界面交互方面。然而,现有方法高度依赖反复的LLM驱动HTML解析,计算成本高且易出错,尤其在动态网页和多步骤任务中表现不佳。本文提出PAFFA(Premeditated Actions For Fast Agents),一种无需任务特定训练的新型推理阶段技术,通过利用基础LLM的参数化知识,预先构建‘动作库’,以生成跨任务通用的浏览器交互模式。通过策略性复用LLM推理:采用‘Dist-Map’实现任务无关的关键交互元素识别,或使用‘Unravel’进行首次接触、有状态的新任务/站点探索,使推理令牌消耗减少87%,同时保持稳健性能(步骤准确率提升至0.57,优于基线0.50)。此外,Unravel可根据探索结果更新动作库,实现对未见过网站的泛化与自适应。本研究表明,LLM推理序列可跨提示泛化,为实现互联网规模数据的次线性令牌消耗推理技术提供了可行路径。
原文摘要 · Abstract (English)
Modern AI assistants have made significant progress in natural language understanding and tool-use, with emerging efforts to interact with Web interfaces. However, current approaches that heavily rely on repeated LLM-driven HTML parsing are computationally expensive and error-prone, particularly when handling dynamic web interfaces and multi-step tasks. We introduce PAFFA (Premeditated Actions For Fast Agents), a method that makes LLMs faster and more accurate in completing tasks on the internet using a novel inference-time technique that requires no task-specific training. PAFFA constructs an 'Action Library', leveraging the parametric knowledge of the base LLM to pre-compute browser interaction patterns that generalize across tasks. By strategically re-using LLM inference across tasks - either via 'Dist-Map' for task-agnostic identification of key interactive web elements, or 'Unravel' for first-encounter, stateful exploration of novel tasks/sites) - PAFFA drastically reduces inference time tokens by 87% while maintaining robust performance (achieving 0.57 vs. 0.50 step accuracy compared to baseline). Further, Unravel's ability to update its action library based on explorations allows generalization and adaptation to unseen websites. In sum, this work exhibits that LLM reasoning sequences can generalize across prompts, offering a way to scale inference-time techniques for internet-scale data with sublinear token count.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。