融合车路协同感知与语言推理，提升自动驾驶安全与效率
V2X-UniPool: Unifying Multimodal Perception and Knowledge Reasoning for Autonomous Driving
- 将多模态车路协同数据转为结构化语言知识，构建时间索引知识池
- 在DAIR-V2X数据集上实现顶尖规划精度，通信开销降低超80%
- 适合关注智能网联汽车、可信决策系统的研究者
自动驾驶虽取得显著进展，但单车感知受限于探测范围与遮挡问题。车路协同（V2X）通信可突破这些限制，却面临异构性、同步与延迟挑战。语言模型具备强知识驱动推理能力，但无法直接处理原始传感器数据且易产生幻觉。我们提出V2X-UniPool，首个统一车路协同感知与语言推理的框架。该框架将多模态V2X数据转化为结构化语言知识，按时间索引组织于知识池中，支持时序一致推理，并采用检索增强生成（RAG）确保决策实时落地。在真实世界DAIR-V2X数据集上的实验表明，V2X-UniPool达到当前最优规划准确率与安全性，同时通信成本降低超过80%，优于所有对比方法。结果验证了融合车路协同感知与语言推理在实现可扩展、可信自动驾驶中的潜力。代码已开源：https://github.com/Xuewen2025/V2X-UniPool
原文摘要 · Abstract (English)
Autonomous driving (AD) has achieved significant progress, yet single-vehicle perception remains constrained by sensing range and occlusions. Vehicle-to-Everything (V2X) communication addresses these limits by enabling collaboration across vehicles and infrastructure, but it also faces heterogeneity, synchronization, and latency constraints. Language models offer strong knowledge-driven reasoning and decision-making capabilities, but they are not inherently designed to process raw sensor streams and are prone to hallucination. We propose V2X-UniPool, the first framework that unifies V2X perception with language-based reasoning for knowledge-driven AD. It transforms multimodal V2X data into structured, language-based knowledge, organizes it in a time-indexed knowledge pool for temporally consistent reasoning, and employs Retrieval-Augmented Generation (RAG) to ground decisions in real-time context. Experiments on the real-world DAIR-V2X dataset show that V2X-UniPool achieves state-of-the-art planning accuracy and safety while reducing communication cost by more than 80\%, achieving the lowest overhead among evaluated methods. These results highlight the promise of bridging V2X perception and language reasoning to advance scalable and trustworthy driving. Our code is available at: https://github.com/Xuewen2025/V2X-UniPool
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。