让机器人用自然语言找没见过的物体,同时减少碰撞、提升泛化能力。
OVSegDT: Segmenting Transformer for Open-Vocabulary Object Goal Navigation
- 用分割掩码+文本对齐,精准定位目标物体空间位置。
- 训练样本量减少33%,碰撞次数降为一半,推理仅需130M参数。
- 无需深度图或大模型,适合真实场景部署的轻量级导航系统。
开放词汇物体目标导航要求具身智能体根据自由形式的语言描述到达从未在训练中见过的物体。现有端到端策略在小型仿真数据集上过拟合,虽在训练场景中表现良好,但泛化能力差且常发生碰撞。我们提出OVSegDT,一种轻量级Transformer策略,包含两个协同组件:第一是语义分支,通过目标二值掩码编码器和辅助分割损失函数,将文本目标与空间位置对齐,提供精确的空间线索;第二是提出的熵自适应损失调节机制,一种基于策略熵的每样本调度器,动态平衡模仿学习与强化学习信号,避免人工阶段切换带来的脆弱性。该设计使训练样本复杂度降低33%,碰撞次数减少两倍,同时保持低推理开销(130M参数,仅需RGB输入)。在HM3D-OVON数据集上,模型在未见类别上的表现与已见类别相当,验证集未见类达到40.1%成功率和20.9%SPL,无需深度图、里程计或大型视觉-语言模型,达到当前最优性能。代码已开源。
原文摘要 · Abstract (English)
Open-vocabulary Object Goal Navigation requires an embodied agent to reach objects described by free-form language, including categories never seen during training. Existing end-to-end policies overfit small simulator datasets, achieving high success on training scenes but failing to generalize and exhibiting unsafe behaviour (frequent collisions). We introduce OVSegDT, a lightweight transformer policy that tackles these issues with two synergistic components. The first component is the semantic branch, which includes an encoder for the target binary mask and an auxiliary segmentation loss function, grounding the textual goal and providing precise spatial cues. The second component consists of a proposed Entropy-Adaptive Loss Modulation, a per-sample scheduler that continuously balances imitation and reinforcement signals according to the policy entropy, eliminating brittle manual phase switches. These additions cut the sample complexity of training by 33%, and reduce collision count in two times while keeping inference cost low (130M parameters, RGB-only input). On HM3D-OVON, our model matches the performance on unseen categories to that on seen ones and establishes state-of-the-art results (40.1% SR, 20.9% SPL on val unseen) without depth, odometry, or large vision-language models. Code is available at https://github.com/CognitiveAISystems/OVSegDT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。