用深度信息提升车辆协同感知，让自动驾驶更懂遮挡下的路况。
D2-V2X: Depth-Driven Cooperative V2X Reasoning for Autonomous Driving
- 通过多车与基础设施传感器构建空间感知问答数据集
- 遮挡物识别召回率达24.4%，可见物体定位误差降低77%
- 适合研究自动驾驶协同感知与视觉语言模型融合的学者
单车视觉语言模型受传感器遮挡限制严重。尽管车联网（V2X）系统可缓解此问题，但现有基准缺乏复杂环境下协同推理能力。我们提出D2-V2X，一个基于多模态车与基础设施传感器的时空感知问答-理由-答案（QRA）基准，包含8,500组三元组。同时建立基线模型，将3D LiDAR特征对齐至VLM隐空间。通过强制生成自然语言思维链再输出结构化JSON，模型被迫显式表达空间关系。实验表明，将VLM与协作式LiDAR结合后，遮挡危险物识别召回率达24.4%，远超零样本模型的近零值；可见物体空间估计误差降低77%。尽管模型决策F1得分为53.5，但3D到2D投影仍是当前VLM架构的根本瓶颈。数据、代码与训练模型已公开于https://github.com/KevinRichard1/D2-V2X。
原文摘要 · Abstract (English)
Single-vehicle Vision-Language Models (VLMs) are fundamentally constrained by sensor occlusions. While Vehicle-to-Everything (V2X) systems mitigate this, current benchmarks lack the cooperative reasoning required for resolving ambiguities in complex environments. We introduce D2-V2X, a spatially-aware Question-Rationale-Answer (QRA) benchmark featuring 8,500 triplets derived from multimodal vehicle and infrastructure sensors. We additionally establish a baseline that aligns 3D LiDAR features with the VLM's latent space. By enforcing natural language Chain-of-Thought rationales prior to structured JSON outputs, our model is forced to explicitly articulate spatial relations. Our experiments demonstrate that grounding VLMs in cooperative LiDAR achieves 24.4% recall in identifying occluded hazards compared to near-zero in zero-shot models and reduces spatial estimation error for visible objects by 77% compared to the zero-shot baseline. While the model achieves a functional decision-making F1-score of 53.5, we identify 3D-to-2D projection as a fundamental bottleneck in current VLM architectures, establishing a new baseline for future innovation. Data, code, and trained models available at https://github.com/KevinRichard1/D2-V2X
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。