智能决定何时思考,让导航更高效。
HiRO-Nav: Hybrid ReasOning Enables Efficient Embodied Navigation
- 根据动作熵动态判断是否需要思考,只在关键步骤推理。
- 在CHORES-S数据集上,成功率更高且节省70%以上计算资源。
- 适合追求高效推理的机器人导航研究者与开发者。
基于大模型的具身导航代理能处理复杂多模态环境输入,并在每一步进行有根据的推理以提升长程任务的决策质量。然而,如何智能高效地利用大模型的推理能力仍是关键问题:简单场景应快速反应,复杂场景则需深思熟虑。为此,我们提出首个可自适应判断每一步是否需推理的导航代理——HiRO-Nav,其依据自身动作熵变化来决定。分析发现,仅少量动作具有高熵,且常引导代理进入新场景或关键物体附近;同时,优化高熵动作对任务成功(即Q值)的提升更大。因此,我们设计了混合训练流程:先通过监督微调冷启动,再结合在线强化学习,仅对高熵动作激活推理,显著降低计算开销。在 extsc{CHORES}-$/mathbb{S}$ ObjectNav基准上的大量实验表明,相比密集推理和无推理基线,HiRO-Nav实现了成功率与令牌效率之间的更优平衡。
原文摘要 · Abstract (English)
Embodied navigation agents built upon large reasoning models (LRMs) can handle complex, multimodal environmental input and perform grounded reasoning per step to improve sequential decision-making for long-horizon tasks. However, a critical question remains: \textit{how can the reasoning capabilities of LRMs be harnessed intelligently and efficiently for long-horizon navigation tasks?} In simple scenes, agents are expected to act reflexively, while in complex ones they should engage in deliberate reasoning before acting.To achieve this, we introduce \textbf{H}ybr\textbf{i}d \textbf{R}eas\textbf{O}ning \textbf{Nav}igation (\textbf{HiRO-Nav}) agent, the first kind of agent capable of adaptively determining whether to perform thinking at every step based on its own action entropy. Specifically, by examining how the agent's action entropy evolves over the navigation trajectories, we observed that only a small fraction of actions exhibit high entropy, and these actions often steer the agent toward novel scenes or critical objects. Furthermore, studying the relationship between action entropy and task completion (i.e., Q-value) reveals that improving high-entropy actions contributes more positively to task success.Hence, we propose a tailored training pipeline comprising hybrid supervised fine-tuning as a cold start, followed by online reinforcement learning with the proposed hybrid reasoning strategy to explicitly activate reasoning only for high-entropy actions, significantly reducing computational overhead while improving decision quality. Extensive experiments on the \textsc{CHORES}-$\mathbb{S}$ ObjectNav benchmark showcases that HiRO-Nav achieves a better trade-off between success rates and token efficiency than both dense-thinking and no-thinking baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。