arXiv:2603.17351cs.RO2026-03被引 1

让机器人在室内外精准导航,用全景感知+高效推理解决视野窄、信息过载难题。

OmniVLN: Omnidirectional 3D Perception and Token-Efficient LLM Reasoning for Visual-Language Navigation across Air and Ground Platforms

  • 融合旋转激光雷达与全景视觉,构建分层动态场景图,实现全局空间理解。
  • 空间指代准确率提升至93.18%,提示词用量减少61.7%,导航成功率提高11.68%。
  • 适合多平台智能体导航研究,尤其适用于复杂室内环境中的视觉语言任务。

语言引导的具身导航要求智能体理解物体参照指令,在多房间间搜索目标并执行可靠运动。现有系统在真实室内环境中受限于窄视野感知,每次仅能获取局部场景,常导致重复旋转、延迟发现目标,并产生碎片化空间认知;同时直接向大模型输入密集3D地图或完整物体列表会迅速超出上下文容量。本文提出OmniVLN,一种零样本视觉语言导航框架,结合全向3D感知与高效的分层推理,适用于空地多平台。OmniVLN将旋转激光雷达与全景视觉融合为硬件无关的地图构建模块,从网格几何逐层构建五层动态场景图(DSG),涵盖房间级到建筑级结构,并通过基于持久同调的房间分割和混合几何/VLM关系验证稳定高层拓扑。导航时,全局DSG转化为以智能体为中心的多分辨率3D八分体表示,结合多尺度空间注意力提示,使大模型逐步筛选候选房间、推断自身朝向、定位目标物体并生成可执行动作,同时保持精细局部细节与紧凑长程记忆。实验表明,该分层接口将空间指代准确率从77.27%提升至93.18%,在杂乱多房间场景中提示词总量减少达61.7%,导航成功率相比平列表基线最高提升11.68%。代码与全向多模态数据集将公开,支持可复现研究。

原文摘要 · Abstract (English)

Language-guided embodied navigation requires an agent to interpret object-referential instructions, search across multiple rooms, localize the referenced target, and execute reliable motion toward it. Existing systems remain limited in real indoor environments because narrow field-of-view sensing exposes only a partial local scene at each step, often forcing repeated rotations, delaying target discovery, and producing fragmented spatial understanding; meanwhile, directly prompting LLMs with dense 3D maps or exhaustive object lists quickly exceeds the context budget. We present OmniVLN, a zero-shot visual-language navigation framework that couples omnidirectional 3D perception with token-efficient hierarchical reasoning for both aerial and ground robots. OmniVLN fuses a rotating LiDAR and panoramic vision into a hardware-agnostic mapping stack, incrementally constructs a five-layer Dynamic Scene Graph (DSG) from mesh geometry to room- and building-level structure, and stabilizes high-level topology through persistent-homology-based room partitioning and hybrid geometric/VLM relation verification. For navigation, the global DSG is transformed into an agent-centric 3D octant representation with multi-resolution spatial attention prompting, enabling the LLM to progressively filter candidate rooms, infer egocentric orientation, localize target objects, and emit executable navigation primitives while preserving fine local detail and compact long-range memory. Experiments show that the proposed hierarchical interface improves spatial referring accuracy from 77.27\% to 93.18\%, reduces cumulative prompt tokens by up to 61.7\% in cluttered multi-room settings, and improves navigation success by up to 11.68\% over a flat-list baseline. We will release the code and an omnidirectional multimodal dataset to support reproducible research.

视觉语言导航三维感知大模型推理机器人导航

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。