arXiv:2607.22854cs.AI2026-07中稿 · USENIX NSDI 2027

让设备端智能体实时通信更流畅,兼顾视频质量与响应速度。

Coordinated Networking for On-Device Agent-Augmented Real-Time Communication

论文配图:Coordinated Networking for On-Device Agent-Augmented Real-Time Communication
图 1 · 摘自论文原文
  • 用统一调度器协同控制视频与智能体数据流发送速率
  • 视频质量提升1.5倍,智能体响应延迟降低31%
  • 适合注重隐私和实时性的智能协作类应用

AI智能体正推动实时通信(RTC)新范式,使人类专注高层协作,智能体则自主实时检索、分析并生成信息以支持互动。例如,在企业员工共同撰写法律文件时,其智能体可代为讨论与起草,减轻手动审阅负担。现有云端智能体存在隐私风险与服务器成本不可扩展问题,设备端智能体增强的RTC成为可行替代方案。但该模式带来新挑战:人类产生的实时视频流与智能体发送上下文文件的流量并发竞争网络资源。为此,我们设计了HFS框架,通过应用层引导的多流传输机制,使统一的上层协调器根据不同需求联合调控视频与智能体数据流的发送速率,确保高质量视频与低延迟响应。基于WebRTC与llama.cpp的原型系统表明,HFS优于基线方法,视频质量提升1.5倍,智能体响应时间减少31%。

原文摘要 · Abstract (English)

AI agents are enabling a new paradigm of agent-augmented real-time communication (RTC), where humans focus on high-level collaboration, while agents autonomously retrieve, analyze, and generate information in real time to support their interactions. These apps enable new experiences across various domains: for example, when corporate employees co-author a legal document, their agents can discuss and draft on their behalf, sparing them the burden of manually reviewing each other's work. As existing cloud-based agents suffer from privacy risks and unscalable server costs, on-device agent-augmented RTC offers a promising alternative. However, this on-device paradigm introduces a new networking challenge: contention between concurrent traffic flows generated by humans (for live video streaming) and agents (for sending context files for analysis). We design HFS, a framework to ensure both high live video quality and low agent response latency in agent-augmented RTC apps. We achieve the goal through an app-guided multi-flow transport approach, where a unified app-layer orchestrator jointly controls the sending rates of live video and agent context flows based on their heterogeneous app requirements. Our prototype built atop WebRTC and llama.cpp demonstrates that HAFS outperforms baselines, achieving 1.5x higher video quality while reducing agent response time by 31%.

实时通信智能体设备端网络调度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。