让网页智能体更省成本：自动判断何时教、教哪几步
When and What to Teach: Budget-Aware Online Adaptation for Web Agents
- 根据可解性决定何时向强模型求助，避免无效提问
- 只保留有信息量的执行步骤，减少52.1%训练算力消耗
- 适合资源有限但需持续优化网页自动化任务的开发者
网页智能体在自动化复杂网络任务上已取得显著进展，但实际部署需持续在线适应。由于强大专有模型部署成本过高，从业者必须依赖轻量本地模型，并通过部署后从强教师模型获取在线教学来进化。然而，标准交互反馈代价高昂。我们发现传统轨迹级偏好优化在不可解回合和冗余执行步骤上浪费预算。为此，提出「预算感知的评分引导在线教学」框架，系统化决策「何时」和「教什么」。具体而言，集成可解性感知教师门控以决定「何时」查询教师模型，以及评分引导的回合选择机制以确定「保留哪些有信息量的步骤」。在MiniWoB和TimeWarp上的大量实验表明，该方法在首通成功率相当的前提下，平均减少22.6%的教师调用次数与52.1%的学生训练计算量。代码已开源。
原文摘要 · Abstract (English)
Web agents have achieved significant success in automating complex internet tasks but deploying them in real-world environments requires continuous online adaptation. Given that deploying powerful proprietary models remains commercially cost-prohibitive, practitioners must rely on lightweight local models that evolve post-deployment via online teaching from a stronger teacher. However, standard interactive feedback imposes prohibitive costs. We show that conventional trajectory-level preference optimization wastes budget on both unresolvable episodes and redundant execution turns. To resolve these inefficiencies, we propose \textbf{Score-Guided Online Teaching with Budgeted Trajectory Trimming}, a budget-aware framework that systematically orchestrates \textbf{when} and \textbf{what} to teach. Specifically, our framework integrates a solvability-aware teacher gate to dictate \textbf{when} to query the teacher model and a score-guided turn selection mechanism to decide \textbf{what} informative turns to retain. Extensive experiments on MiniWoB and TimeWarp demonstrate that our method achieves comparable first-pass success while reducing teacher calls by 22.6\% and student training compute by 52.1\% on average. Our code is available at https://github.com/zjw131f1fc/budgeted-online-teaching.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。