arXiv:2605.24598cs.AIcs.MA2026-05被引 2

Hera让大模型在设备与云端间智能协作,省时省力还高效。

Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agents

论文配图:Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agents
图 1 · 摘自论文原文
  • 按每一步动态决定用设备还是云端,比传统方法更灵活。
  • 在三个任务中达成92.5%的云端成功率,仅46.3%步骤调用云端。
  • 适合需要长期任务、兼顾性能与成本的智能系统部署者。

大型语言模型(LLM)代理通过与环境自主交互,在解决复杂长时程任务上表现出色。然而其真实部署面临设备-云困境:本地模型高效但脆弱,云端模型强大却计算开销高。现有设备-云路由通常只做粗粒度的任务级决策,难以适应多步交互中难度的变化。为此,我们提出Hera,一种面向长时程任务的步级设备-云协调器,实现了性能与成本的帕累托最优。Hera采用新颖的两阶段训练范式:(1) 模仿学习用于冷启动,(2) 成本感知强化学习联合优化任务成功率与云端使用效率。第一阶段将步级路由视为监督分类问题:在云端轨迹上回放设备代理,每个状态根据设备与云端动作的一致性打标签。第二阶段通过聚合轨迹中相同状态,并以提升预期回报和减少未来云端调用为目标更新Hera。我们在ALFWorld、WebShop和AppWorld上评估Hera,结果一致优于先前方法,达到云端全用时92.5%的成功率,且仅在46.3%的步骤中调用云端。

原文摘要 · Abstract (English)

Large language model (LLM) agents excel at solving complex long-horizon tasks through autonomous interaction with environments. However, their real-world deployment faces a fundamental device--cloud dilemma: on-device models are efficient but often brittle, while cloud models are stronger but costly in computation. State-of-the-art LLM device--cloud routers usually make coarse task-level decisions, which cannot adapt to the changing difficulty of multi-step agent interactions. To address this issue, we present Hera, a step-level device--cloud LLM agent coordinator for long-horizon tasks achieving a strong performance--cost Pareto frontier. Hera adopts a novel two-stage training paradigm: (1) imitation learning for cold-start, followed by (2) reinforcement learning that jointly optimizes task success and cloud usage efficiency. The first stage casts step-level routing as a supervised classification problem: the device agent is replayed on cloud trajectories, with each state labeled by the agreement between device and cloud actions. In the second stage, we perform cost-aware reinforcement learning by grouping identical states across trajectories and updating Hera with labels favoring higher expected return and fewer future cloud calls. We evaluate Hera on ALFWorld, WebShop, and AppWorld, where it consistently outperforms prior methods, achieving 92.5% of the cloud-only success rate with cloud use in only 46.3% of steps.

大模型设备云协同长时程任务强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。