用多模态序列感知实现足式机器人端到端导航,提升复杂地形适应性。
Seq-DeepIPC: Sequential Sensing for End-to-End Control in Legged Robot Navigation
- 融合RGB-D与GNSS的时序感知,联合预测语义分割与深度图
- 通过序列化GNSS差分计算航向,省去易噪陀螺仪,降低计算开销
- 在真实道路与草地场景中验证,适合边缘设备部署,适合足式机器人研究者
我们提出Seq-DeepIPC,一种用于真实环境足式机器人导航的序列化端到端感知-控制模型。该模型通过紧密集成多模态感知(RGB-D + GNSS)与时间融合技术,实现感知与控制的统一。模型联合预测语义分割与深度估计,提供更丰富的空间特征以支持规划与控制。为适配边缘设备部署,采用轻量编码器,在保持精度的同时减少计算量。航向估计通过序列化GNSS坐标差分分析实现,无需使用噪声较大的惯性测量单元(IMU)。我们构建了更大更多样化的数据集,涵盖道路与草地地形,并在机器狗上验证了模型性能。对比与消融实验表明,序列输入显著提升感知与控制表现,而其他基线方法未获益。Seq-DeepIPC在合理模型规模下达到竞争性或更优结果;尽管在高楼密集区域GNSS航向可靠性下降,但在开阔区域表现稳健。总体而言,该工作将端到端导航从轮式机器人拓展至更灵活、具备时序感知能力的足式系统。代码将开源至https://github.com/oskarnatan/Seq-DeepIPC,支持后续研究。
原文摘要 · Abstract (English)
We present Seq-DeepIPC, a sequential end-to-end perception-to-control model for legged robot navigation in real-world environments. Seq-DeepIPC advances intelligent sensing for autonomous legged navigation by tightly integrating multi-modal perception (RGB-D + GNSS) with temporal fusion and control. The model jointly predicts semantic segmentation and depth estimation, giving richer spatial features for planning and control. For efficient deployment on edge devices, we use a lightweight model as the encoder, reducing computation while maintaining accuracy. Heading estimation is simplified by removing the noisy IMU and instead deriving global heading via differential analysis of sequential GNSS coordinates. We collected a larger and more diverse dataset that includes both road and grass terrains, and validated Seq-DeepIPC on a robot dog. Comparative and ablation studies show that sequential inputs improve perception and control in our models, while other baselines do not benefit. Seq-DeepIPC achieves competitive or better results with reasonable model size; although GNSS-only heading is less reliable near tall buildings, it is robust in open areas. Overall, Seq-DeepIPC extends end-to-end navigation beyond wheeled robots to more versatile and temporally-aware systems. To support future research, we will release the codes to our GitHub repo at https://github.com/oskarnatan/Seq-DeepIPC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。