用最优传输融合多模态信息,提升越野路段在分布外场景的分割准确率。
OT-Drive: Out-of-Distribution Off-Road Traversable Area Segmentation via Optimal Transport
- 通过最优传输将图像与表面法线特征对齐到语义锚点空间。
- 在ORFD数据集上达到95.16% mIoU,比之前方法高6.35%。
- 仅需少量训练数据即可实现强泛化,适合真实自动驾驶部署。
在非结构化环境中的可靠可通行区域分割对自动驾驶规划与决策至关重要。然而,现有数据驱动方法在分布外(OOD)场景下性能下降,影响下游任务。为此,我们提出OT-Drive,一种基于最优传输的多模态融合框架。该方法将RGB图像与表面法线特征融合建模为分布传输问题。具体地,设计新型场景锚点生成器(SAG),将场景信息分解为天气、时间与道路类型联合分布,构建可泛化至未见场景的语义锚点。随后,设计基于最优传输的多模态融合模块(OT Fusion),将RGB与表面法线特征传输至语义锚点定义的流形上,实现分布外场景下的鲁棒可通行区域分割。实验表明,该方法在ORFD OOD场景中达到95.16% mIoU,优于先前方法6.35%;在跨数据集迁移任务中达89.79% mIoU,超越基线13.99%。结果表明,所提模型仅需少量训练数据即可实现强泛化能力,显著提升实际部署的实用性与效率。
原文摘要 · Abstract (English)
Reliable traversable area segmentation in unstructured environments is critical for planning and decision-making in autonomous driving. However, existing data-driven approaches often suffer from degraded segmentation performance in out-of-distribution (OOD) scenarios, consequently impairing downstream driving tasks. To address this issue, we propose OT-Drive, an Optimal Transport--driven multi-modal fusion framework. The proposed method formulates RGB and surface normal fusion as a distribution transport problem. Specifically, we design a novel Scene Anchor Generator (SAG) to decompose scene information into the joint distribution of weather, time-of-day, and road type, thereby constructing semantic anchors that can generalize to unseen scenarios. Subsequently, we design an innovative Optimal Transport-based multi-modal fusion module (OT Fusion) to transport RGB and surface normal features onto the manifold defined by the semantic anchors, enabling robust traversable area segmentation under OOD scenarios. Experimental results demonstrate that our method achieves 95.16% mIoU on ORFD OOD scenarios, outperforming prior methods by 6.35%, and 89.79% mIoU on cross-dataset transfer tasks, surpassing baselines by 13.99%.These results indicate that the proposed model can attain strong OOD generalization with only limited training data, substantially enhancing its practicality and efficiency for real-world deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。