Astra让机器人在复杂室内环境自主导航，靠双模型架构提升适应性。
Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal Learning
- 用双模型结构：全局大模型处理视觉语言，局部网络做路径规划与定位
- 实测在多种室内场景中达成高成功率，优于传统视觉定位方法
- 适合需要自适应导航的移动机器人研发人员参考
现代机器人导航系统在多样复杂的室内环境中面临挑战。传统方法依赖多个小型模型或基于规则的模块，难以适应新环境。为此，我们开发了Astra，一种面向移动机器人导航的双模型架构，包括Astra-Global和Astra-Local。Astra-Global是一个多模态大语言模型，结合视觉与语言输入，利用混合拓扑-语义图作为全局地图，实现自我与目标定位，性能优于传统视觉场景识别方法。Astra-Local是一个多任务网络，负责局部路径规划与里程计估计。其4D时空编码器通过自监督学习训练，生成鲁棒的4D特征用于下游任务。规划头采用流匹配与新型掩码ESDF损失，有效降低碰撞风险以生成局部轨迹；里程计头则通过Transformer编码器融合多传感器输入，预测机器人相对位姿。在真实家用移动机器人上部署后，Astra在多种室内环境中实现了高端到端任务成功率。
原文摘要 · Abstract (English)
Modern robot navigation systems encounter difficulties in diverse and complex indoor environments. Traditional approaches rely on multiple modules with small models or rule-based systems and thus lack adaptability to new environments. To address this, we developed Astra, a comprehensive dual-model architecture, Astra-Global and Astra-Local, for mobile robot navigation. Astra-Global, a multimodal LLM, processes vision and language inputs to perform self and goal localization using a hybrid topological-semantic graph as the global map, and outperforms traditional visual place recognition methods. Astra-Local, a multitask network, handles local path planning and odometry estimation. Its 4D spatial-temporal encoder, trained through self-supervised learning, generates robust 4D features for downstream tasks. The planning head utilizes flow matching and a novel masked ESDF loss to minimize collision risks for generating local trajectories, and the odometry head integrates multi-sensor inputs via a transformer encoder to predict the relative pose of the robot. Deployed on real in-house mobile robots, Astra achieves high end-to-end mission success rate across diverse indoor environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。