通过任务提示与空间推理，提升视觉语言模型在自动驾驶中的表现。
Enhancing Vision-Language Models for Autonomous Driving through Task-Specific Prompting and Spatial Reasoning
- 用提示路由系统区分任务，分配专用提示模板避免干扰。
- 在干净数据上达70.87%准确率，污染数据上达72.85%。
- 适合关注自动驾驶多任务理解与提示工程的研究者。
本技术报告介绍了我们在IROS 2025 RoboSense挑战赛中的解决方案，评估视觉语言模型（VLMs）在自动驾驶场景理解中的表现，涵盖感知、预测、规划和异常检测任务。我们提出一个包含四个核心组件的系统：首先，提示混合路由器根据问题类型分类并分发至特定任务的专家提示，消除不同任务间的干扰；其次，每个任务使用嵌入坐标系、空间推理规则、角色扮演、思维链/树等结构化提示及少样本示例；第三，视觉组装模块依据问题需求融合多视角图像、目标裁剪、品红标记与自适应历史帧；第四，针对各任务配置推理参数（温度、top-p、消息角色）以优化输出质量。基于Qwen2.5-VL-72B模型，该方法在第一阶段（清洁数据）取得70.87%平均准确率，在第二阶段（污染数据）达到72.85%，表明结构化提示与空间对齐显著提升VLM在高安全要求自动驾驶任务中的性能。代码与提示已开源：https://github.com/wuaodi/UCAS-CSU-phase2。
原文摘要 · Abstract (English)
This technical report presents our solution for the RoboSense Challenge at IROS 2025, which evaluates Vision-Language Models (VLMs) on autonomous driving scene understanding across perception, prediction, planning, and corruption detection tasks. We propose a systematic framework built on four core components. First, a Mixture-of-Prompts router classifies questions and dispatches them to task-specific expert prompts, eliminating interference across diverse question types. Second, task-specific prompts embed explicit coordinate systems, spatial reasoning rules, role-playing, Chain-of-Thought/Tree-of-Thought reasoning, and few-shot examples tailored to each task. Third, a visual assembly module composes multi-view images with object crops, magenta markers, and adaptive historical frames based on question requirements. Fourth, we configure model inference parameters (temperature, top-p, message roles) per task to optimize output quality. Implemented on Qwen2.5-VL-72B, our approach achieves 70.87% average accuracy on Phase-1 (clean data) and 72.85% on Phase-2 (corrupted data), demonstrating that structured prompting and spatial grounding substantially enhance VLM performance on safety-critical autonomous driving tasks. Code and prompt are available at https://github.com/wuaodi/UCAS-CSU-phase2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。