提出鲁棒多模态路由框架,实现越野环境下的场景描述与路径规划。
Wild-Drive: Off-Road Scene Captioning and Path Planning via Robust Multi-modal Routing and Efficient Large Language Model
- 设计MoRo-Former路由模块,动态融合可靠感知信息。
- 在复杂天气下仍保持稳定性能,优于现有基于LLM的方法。
- 适用于野外自动驾驶系统,提升安全性和可解释性。
可解释性与透明决策对自动驾驶系统的安全部署至关重要。场景描述以自然语言总结环境状态与风险因素,提升透明度、安全性及人机交互。然而,现有方法主要针对城市结构化场景;在越野环境中,它们易受雨、雾、雪、黑暗等导致的单模态退化影响,且缺乏统一建模结构化场景描述与路径规划的框架。为此,本文提出Wild-Drive,一个面向越野场景的高效场景描述与路径规划框架。该框架采用现代多模态编码器,引入任务条件的模态路由桥接模块MoRo-Former,以在传感退化条件下自适应聚合可靠信息。随后结合高效大语言模型(LLM),通过规划标记与门控循环单元(GRU)解码器,生成结构化描述并预测未来轨迹。我们还构建了OR-C2P基准数据集,涵盖多种传感器退化条件下的越野场景描述与路径规划任务。在OR-C2P数据集和自采数据集上的实验表明,Wild-Drive优于先前基于LLM的方法,且在传感退化下更具稳定性。代码与基准将公开于https://github.com/wangzihanggg/Wild-Drive。
原文摘要 · Abstract (English)
Explainability and transparent decision-making are essential for the safe deployment of autonomous driving systems. Scene captioning summarizes environmental conditions and risk factors in natural language, improving transparency, safety, and human--robot interaction. However, most existing approaches target structured urban scenarios; in off-road environments, they are vulnerable to single-modality degradations caused by rain, fog, snow, and darkness, and they lack a unified framework that jointly models structured scene captioning and path planning. To bridge this gap, we propose Wild-Drive, an efficient framework for off-road scene captioning and path planning. Wild-Drive adopts modern multimodal encoders and introduces a task-conditioned modality-routing bridge, MoRo-Former, to adaptively aggregate reliable information under degraded sensing. It then integrates an efficient large language model (LLM), together with a planning token and a gate recurrent unit (GRU) decoder, to generate structured captions and predict future trajectories. We also build the OR-C2P Benchmark, which covers structured off-road scene captioning and path planning under diverse sensor corruption conditions. Experiments on OR-C2P dataset and a self-collected dataset show that Wild-Drive outperforms prior LLM-based methods and remains more stable under degraded sensing. The code and benchmark will be publicly available at https://github.com/wangzihanggg/Wild-Drive.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。