arXiv:2607.12965cs.RO2026-07中稿 · IROS 2026 Main Con…

多模态融合的自动驾驶系统,缺啥模态都能稳行

MAMMOTH: A Multi-Modal End-to-End Policy for Off-Road Mobility Robust to Missing Modality

论文配图:MAMMOTH: A Multi-Modal End-to-End Policy for Off-Road Mobility Robust to Missing Modality
图 1 · 摘自论文原文
  • 用多模态输入+丢弃训练,让模型适应传感器失效
  • 夜间和复杂光照下碰撞率降低47%,路径更安全平滑
  • 适合野外无人车、机器人等极端环境导航任务

在非结构化野外环境中实现可靠的自主导航仍是重大挑战,主要源于地形多样、光照剧烈变化及传感器性能退化。现有方法多依赖RGB图像,在强光、阴影或低照度下表现差。为提升鲁棒性,本文提出MAMMOTH(MAsking Multi-Modal inputs for Off-road Traversability Heuristic-informed navigation),一种统一的端到端导航策略,融合RGB、热成像、3D点云与自车速度信息。通过模态丢弃训练,模型可在推理时应对任意模态缺失。采用扩散策略学习物理可行轨迹的联合条件分布,并引入可通行性启发式,偏好更安全平滑路径。在多个真实野外场景(含夜间)的实机测试中,显著提升避障能力与地形感知精度,对缺失模态具备强泛化能力。代码与数据集将公开。

原文摘要 · Abstract (English)

Reliable autonomous navigation in unstructured off-road environments remains a critical unsolved challenge due to extreme terrain diversity, drastic illumination variations and acute sensor degradation. Recent developments have approached the problem as a traversability costmap estimation or visual navigation task. However, many exhibit heavy reliance on RGB modality, leading to poor performance in varied illumination such as glares, shadows or low ambient light. Achieving robust generalization in such conditions requires integrating modalities that provide supplementary scene information. Such multi-modal methods suffer from a rigid dependency on the presence of near-perfect sensor inputs, leaving them unable to robustly handle sensor degradation or individual modality failure. To address these limitations, we introduce MAMMOTH (MAsking Multi-Modal inputs for Off-road Traversability Heuristic-informed navigation), a unified end-to-end navigation policy for robust off-road visual-goal-conditioned navigation and undirected exploration. Specifically, MAMMOTH efficiently fuses multi-modal observations (RGB, Thermal, 3D Pointcloud and Ego Velocity) and is trained with a modality dropout scheme, enabling it to generalize to missing modalities at inference time. Furthermore, we employ a diffusion policy to learn the joint conditional probability distribution of physically-grounded trajectories and a intrinsic traversability heuristic. MAMMOTH utilizes this heuristic to prefer safer, smoother trajectories. We validate MAMMOTH through extensive real-world robot experiments in distinct off-road environments, including night-time operation. Our results demonstrate superior performance, with significant improvements in collision avoidance, terrain-aware planning and generalization to missing modalities. The code and dataset used for this work will be made publicly available.

多模态自动驾驶机器人视觉导航

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。