人机协作网页导航框架，显著减少人工操作步骤。
CowPilot: A Framework for Autonomous and Human-Agent Collaborative Web Navigation
- 人机协同控制，代理提出下一步，人类可干预或接管
- 在五网站测试中成功率95%，人类仅需执行15.2%步骤
- 适合研究人机协作、网页自动化及智能代理评估
尽管现有网页代理研究侧重于自主完成用户任务，但在真实复杂场景中，代理在任务执行和用户偏好建模方面仍存在不足。这为人类与代理协同提供了机会。我们提出CowPilot框架，支持完全自主及人机协同的网页导航，并在任务成功率与效率上进行评估。该框架通过让代理提议下一步操作，使人类只需执行少量操作；用户可在执行中随时暂停、拒绝或采取替代行动，也可覆盖代理建议或重新启用代理控制。我们在五个常见网站上开展案例研究，发现人机协同模式下任务成功率达95%，人类仅需执行总步骤的15.2%。即使有干预，代理仍能独立驱动任务成功的一半以上。CowPilot可作为跨网站数据收集与代理评估工具，推动人机协作机制研究。视频演示见https://oaishi.github.io/cowpilot.html。
原文摘要 · Abstract (English)
While much work on web agents emphasizes the promise of autonomously performing tasks on behalf of users, in reality, agents often fall short on complex tasks in real-world contexts and modeling user preference. This presents an opportunity for humans to collaborate with the agent and leverage the agent's capabilities effectively. We propose CowPilot, a framework supporting autonomous as well as human-agent collaborative web navigation, and evaluation across task success and task efficiency. CowPilot reduces the number of steps humans need to perform by allowing agents to propose next steps, while users are able to pause, reject, or take alternative actions. During execution, users can interleave their actions with the agent by overriding suggestions or resuming agent control when needed. We conducted case studies on five common websites and found that the human-agent collaborative mode achieves the highest success rate of 95% while requiring humans to perform only 15.2% of the total steps. Even with human interventions during task execution, the agent successfully drives up to half of task success on its own. CowPilot can serve as a useful tool for data collection and agent evaluation across websites, which we believe will enable research in how users and agents can work together. Video demonstrations are available at https://oaishi.github.io/cowpilot.html
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。