让机器人在360度视野中准确定向,无需训练就能自适应运动变化。
EAGOR: Embodied Reasoning in Omni-direction

- 直接在球面上做方向推理,用球谐函数表示信念分布。
- 在真实机器人上测试,导航成功率提升14.6%,方向误差降低24.5%。
- 不依赖训练,适合无地图导航等对方向一致性要求高的任务。
全景(360°)摄像头为具身智能体提供全局环境视图,适用于导航与目标搜索等方向性任务。现有视觉语言模型将360°观测投影为2D等距柱状图(ERP),并使用针对透视图像设计的架构处理,忽视了360°观测的球面本质——每个像素代表相对于智能体的观察方向。因此,在智能体运动导致视角变换时,其方向估计常出现不一致。这一缺陷在无地图导航中尤为关键,需持续在自身坐标系中估计目标方向。我们提出EAGOR,一种无需训练、几何感知的360°方向推理框架。不将目标方向预测为ERP图像坐标,而是将方向推理建模为球面上的递归贝叶斯估计,维护连续的方向信念,并在智能体运动下等变传播,无需微调主干VLM。为此引入球谐信念场(SH-BF),其球谐表示提供了球面流形上全局定义、旋转敏感的方向估计基底。该方法消除了ERP接缝不连续、纬度畸变和插值误差。我们在两个基准数据集及真实腿式机器人实验中评估EAGOR,结果表明其性能持续优于现有方法,在HOS和OSR-Bench上平均相对提升分别为+34.4%和+45.6%,导航成功率提升+14.6%,步数减少17.7%,平均角度误差降低24.5%。
原文摘要 · Abstract (English)
Omni-directional (360°) cameras provide embodied agents with a holistic view of their surroundings, making them suited for directional reasoning in tasks such as navigation and object search. Existing Vision Language Models (VLMs) project 360° observations to 2D equirectangular projection (ERP) images and process them using architectures designed for perspective images. However, they ignore the spherical nature of 360° observations, where each pixel represents a viewing direction relative to the agent. Consequently, their direction estimates often become inconsistent under camera view transformations caused by agent motion. This limitation is particularly critical for map-free navigation, where the agent must continuously estimate the target direction in its egocentric frame. We propose EAGOR, a training-free, geometry-aware framework for embodied 360° directional reasoning. Instead of predicting target directions as ERP image coordinates, EAGOR formulates directional reasoning as recursive Bayesian estimation directly on the sphere. It maintains a continuous belief over target directions and propagates it equivariantly under agent motion without training the backbone VLMs. To achieve this, we introduce the Spherical Harmonic Belief Field (SH-BF), whose spherical harmonic representation provides a globally defined, rotation-aware basis for directional estimation on the spherical manifold. This formulation eliminates ERP seam discontinuities, latitude distortions, and interpolation errors. We evaluate EAGOR on two benchmark datasets and real-world experiments with a legged robot across directional reasoning tasks. EAGOR consistently outperforms existing methods, achieving average relative gains of +34.4% and +45.6% on HOS and OSR-Bench, respectively, while improving navigation success by +14.6%, reducing step count by 17.7%, and lowering mean angular error by 24.5%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。