让人类和AI无缝切换控制,提升复杂任务成功率。
AgentBay: A Hybrid Interaction Sandbox for Seamless Human-AI Intervention in Agentic Systems
- 设计统一交互沙箱,支持多平台安全运行。
- 人机协同任务成功率达48%以上提升。
- 自适应流传输协议,低延迟抗弱网。
大型语言模型的快速发展正推动自主AI智能体执行复杂多步任务。然而,面对现实世界异常时,这些智能体仍显脆弱,因此在关键应用中需要人类介入监督。本文提出AgentBay,一种从零构建的混合交互沙箱服务。它提供跨Windows、Linux、Android、Web浏览器及代码解释器的隔离执行环境。核心贡献是通过混合控制界面实现统一会话:AI代理可通过主流接口(MCP、开源SDK)程序化交互,而人类操作员可在任意时刻无缝接管全部手动控制。这一无缝干预由自适应流传输协议(ASP)实现。不同于传统VNC/RDP,ASP专为混合场景设计,可在弱网络下保持超低延迟与流畅体验。其通过动态融合命令流与视频流,根据网络状况和当前控制者(AI或人)自适应编码策略。评估显示,在复杂任务基准测试中,AgentBay(AI+人类)模式任务完成率提升超过48%;相比标准RDP,ASP带宽消耗降低最多50%,端到端延迟减少约5%,尤其在弱网条件下表现更优。我们认为,AgentBay为构建下一代可靠、人类监督的自主系统提供了基础支撑。
原文摘要 · Abstract (English)
The rapid advancement of Large Language Models (LLMs) is catalyzing a shift towards autonomous AI Agents capable of executing complex, multi-step tasks. However, these agents remain brittle when faced with real-world exceptions, making Human-in-the-Loop (HITL) supervision essential for mission-critical applications. In this paper, we present AgentBay, a novel sandbox service designed from the ground up for hybrid interaction. AgentBay provides secure, isolated execution environments spanning Windows, Linux, Android, Web Browsers, and Code interpreters. Its core contribution is a unified session accessible via a hybrid control interface: An AI agent can interact programmatically via mainstream interfaces (MCP, Open Source SDK), while a human operator can, at any moment, seamlessly take over full manual control. This seamless intervention is enabled by Adaptive Streaming Protocol (ASP). Unlike traditional VNC/RDP, ASP is specifically engineered for this hybrid use case, delivering an ultra-low-latency, smoother user experience that remains resilient even in weak network environments. It achieves this by dynamically blending command-based and video-based streaming, adapting its encoding strategy based on network conditions and the current controller (AI or human). Our evaluation demonstrates strong results in security, performance, and task completion rates. In a benchmark of complex tasks, the AgentBay (Agent + Human) model achieved more than 48% success rate improvement. Furthermore, our ASP protocol reduces bandwidth consumption by up to 50% compared to standard RDP, and in end-to-end latency with around 5% reduction, especially under poor network conditions. We posit that AgentBay provides a foundational primitive for building the next generation of reliable, human-supervised autonomous systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。