大模型和小模型协同进化,让网页代理更智能高效。
Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMs
- 大模型生成高质量操作轨迹,小模型通过多任务学习提升推理能力。
- 小模型探索新路径,使数据集持续进化,大模型性能提升至52%。
- 提出隐私保护的混合模式,适合需要安全性的自动化场景。
基于大语言模型(LLM)的网页浏览代理在自动化复杂网络任务方面展现出巨大潜力。现有方法通常依赖大模型(如GPT-4o)探索网页环境并生成轨迹数据,这些数据用于大模型的演示检索或蒸馏小模型(如Llama3),但该过程与探索解耦。本文提出AgentSymbiotic框架,通过迭代式耦合数据合成与任务表现,实现大、小模型的“共生提升”。研究发现:大模型擅长生成高质量蒸馏轨迹,而小模型因推理方式不同,常选择与大模型不同的动作,从而探索新路径,丰富合成数据。然而小模型性能成为瓶颈。为此,我们提出两种创新:推测性数据合成策略以缓解离策略偏差,以及多任务学习方法提升学生模型推理能力。此外,引入混合模式保障用户隐私。在WEBARENA基准上,AgentSymbiotic达到当前最优性能:最佳大模型代理达52%,超越此前最高45%;8B蒸馏模型达49%,远超此前28%。代码将在接受后发布。
原文摘要 · Abstract (English)
Web browsing agents powered by large language models (LLMs) have shown tremendous potential in automating complex web-based tasks. Existing approaches typically rely on large LLMs (e.g., GPT-4o) to explore web environments and generate trajectory data, which is then used either for demonstration retrieval (for large LLMs) or to distill small LLMs (e.g., Llama3) in a process that remains decoupled from the exploration. In this paper, we propose AgentSymbiotic, an iterative framework that couples data synthesis with task-performance, yielding a "symbiotic improvement" for both large and small LLMs. Our study uncovers a complementary dynamic between LLM types: while large LLMs excel at generating high-quality trajectories for distillation, the distilled small LLMs-owing to their distinct reasoning capabilities-often choose actions that diverge from those of their larger counterparts. This divergence drives the exploration of novel trajectories, thereby enriching the synthesized data. However, we also observe that the performance of small LLMs becomes a bottleneck in this iterative enhancement process. To address this, we propose two innovations in LLM distillation: a speculative data synthesis strategy that mitigates off-policy bias, and a multi-task learning approach designed to boost the reasoning capabilities of the student LLM. Furthermore, we introduce a Hybrid Mode for Privacy Preservation to address user privacy concerns. Evaluated on the WEBARENA benchmark, AgentSymbiotic achieves SOTA performance with both LLM types. Our best Large LLM agent reaches 52%, surpassing the previous best of 45%, while our 8B distilled model demonstrates a competitive 49%, exceeding the prior best of 28%. Code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。