用离线轨迹提炼精准提示,让网页代理快速适应新任务。
JEF-Hinter: Leveraging Offline Knowledge for Improving Web Agents Adaptation
- 从长而杂的离线操作中提取关键步骤作为上下文提示。
- 在三个基准上表现优于人类和文档提示,成功率提升12%以上。
- 支持失败数据也能生成有效指导,适合资源受限场景。
大型语言模型代理在序列决策任务中表现良好,但在不熟悉领域提升性能通常需要昂贵的在线交互或大规模专家数据微调。这些方法对闭源模型不适用,对开源模型成本高且存在灾难性遗忘风险。离线轨迹可提供可复用知识,但基于示范的方法因原始轨迹过长、噪声大且绑定特定任务而效果不佳。我们提出即时情景反馈提示器(JEF-Hinter),将离线轨迹提炼为紧凑、上下文感知的提示。通过“聚焦”机制识别长轨迹中的关键决策点,捕捉策略与陷阱。与以往方法不同,JEF-Hinter可同时利用成功与失败轨迹,在仅有失败数据时仍能提取有效指导,支持并行化提示生成与无基准提示。推理时,检索器根据当前状态选择相关提示,提供透明可追溯的针对性引导。在MiniWoB++、WorkArena-L1和WebArena-Lite上的实验表明,JEF-Hinter持续优于强基线,包括人类与文档提示。
原文摘要 · Abstract (English)
Large language model (LLM) agents perform well in sequential decision-making tasks, but improving them on unfamiliar domains often requires costly online interactions or fine-tuning on large expert datasets. These strategies are impractical for closed-source models and expensive for open-source ones, with risks of catastrophic forgetting. Offline trajectories offer reusable knowledge, yet demonstration-based methods struggle because raw traces are long, noisy, and tied to specific tasks. We present Just-in-time Episodic Feedback Hinter (JEF-Hinter), an agentic system that distills offline traces into compact, context-aware hints. A zooming mechanism highlights decisive steps in long trajectories, capturing both strategies and pitfalls. Unlike prior methods, JEF-Hinter leverages both successful and failed trajectories, extracting guidance even when only failure data is available, while supporting parallelized hint generation and benchmark-independent prompting. At inference, a retriever selects relevant hints for the current state, providing targeted guidance with transparency and traceability. Experiments on MiniWoB++, WorkArena-L1, and WebArena-Lite show that JEF-Hinter consistently outperforms strong baselines, including human- and document-based hints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。