arXiv:2512.09706cs.LG2025-12中稿 · CVPR

一个模型自适应切换不同操作层级,提升智能体在复杂环境中的效率与泛化能力。

Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learning

  • 通过强化学习让模型自主选择高阶指令或低阶操作,动态适配任务需求。
  • 在超过800个Minecraft任务中达到当前最优表现,长程推理效率显著提升。
  • 无需人工设定规则,适合需要跨层级协同的智能体研发与应用。

智能体范式正从预设复杂工作流转向后训练的原生模型。然而,现有智能体通常局限于静态、预定义的动作空间,如仅使用API、GUI事件或机器人指令。这种僵化限制了其在动态环境中根据上下文调整交互粒度的能力。为此,我们提出CrossHA,一种统一的智能体模型,能够掌握异构动作空间,并自主选择每一步轨迹中最有效的接口。我们设计了一个完整的训练流程,融合冷启动监督微调与多轮组相对策略优化(GRPO)算法。该方法使智能体学会自适应动作切换,在高效性与精确性之间取得平衡,无需人类指定规则。在开放世界Minecraft环境中的800多个任务上进行的大量实验表明,CrossHA达到当前最优性能。通过动态利用不同动作空间的优势,模型显著优于固定动作基线,展现出更强的泛化能力和长程推理效率。所有代码与模型已开源:https://github.com/CraftJarvis/OpenHA。

原文摘要 · Abstract (English)

The paradigm of agentic AI is shifting from engineered complex workflows to post-training native models. However, existing agents are typically confined to static, predefined action spaces-such as exclusively using APIs, GUI events, or robotic commands. This rigidity limits their adaptability in dynamic environments where the optimal granularity of interaction varies contextually. To bridge this gap, we propose CrossHA, a unified agentic model that masters heterogeneous action spaces and autonomously selects the most effective interface for each step of a trajectory. We introduce a comprehensive training pipeline that integrates cold-start supervised fine-tuning with a Multi-Turn Group Relative Policy Optimization (GRPO) algorithm. This approach enables the agent to learn adaptive action switching-balancing high-level efficiency with low-level precision-without human-specified rules. Extensive experiments on over 800 tasks in the open-world Minecraft environment demonstrate that CrossHA achieves state-of-the-art performance. By dynamically leveraging the strengths of diverse action spaces, our model significantly outperforms fixed-action baselines, exhibiting superior generalization and efficiency in long-horizon reasoning. All code and models are available at https://github.com/CraftJarvis/OpenHA.

智能体强化学习自适应Minecraft

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。