用视觉语言模型实现边缘设备上的物理感知驾驶推理，体积小且准确。
MoRAL: Sensor-Grounded BEV Reasoning for Compact VLMs toward Edge-Oriented Autonomous Driving
- 将激光雷达、雷达数据转为彩色鸟瞰图，让模型直接读图推理
- 仅用5200万参数，7个驾驶问题类型表现超越80亿参数基线
- 适合资源受限的车载系统，42词/秒高效运行在消费级显卡上
在资源受限的自动驾驶平台部署视觉语言模型进行安全关键的空间推理，需兼顾模型紧凑与可靠度量对齐。我们提出MoRAL（面向自主语言模型的多模态推理），一个两阶段微调流程，使Cosmos-Reason2-2B先读取编码物理信息的鸟瞰图（BEV），再据此做出驾驶决策。该BEV图像以颜色带表示激光雷达距离，以聚类形态表示物体类别，以方向楔形叠加表示雷达多普勒速度，将空间感知外化至输入图像，推理时无需学习3D主干网络。第一阶段在6万条对齐记录上微调视觉编码器；零样本基线无法解析出可读的BEV输出，证实词汇需显式训练。第二阶段在57,696条由Cosmos-Reason2-8B生成的链式思维记录上微调完整模型（5200万参数，占总量2.4%），覆盖八类驾驶问题。在2304帧未见nuScenes数据上，经Gemma 4（31B）校准并对比人工评估，尽管参数量仅为基线四分之一，MoRAL在七类问题中胜出，尤其在需要结构化多步物理推理的问题上提升显著：紧急制动召回率从10.8%提升至47.8%，输出退化率从94.1%降至20.8%，整个流水线可在不量化情况下适配消费级8GB GPU，运行速度达42词/秒。结果建立了一个可在移动边缘平台复现的紧凑、物理对齐的VLM推理范式。
原文摘要 · Abstract (English)
Deploying vision-language models (VLMs) for safety-critical spatial reasoning on resource-constrained autonomous driving platforms requires both compact model size and reliable metric grounding. We present MoRAL (Multimodal Reasoning for Autonomous Language Models), a two-stage fine-tuning pipeline that teaches Cosmos-Reason2-2B to first read a physics-encoded Bird's Eye View (BEV) representation and then reason over it for driving decisions. The BEV image encodes LiDAR metric distance as color bands, object class as cluster morphology, and radar Doppler velocity as directional wedge overlays, externalizing spatial perception into the input image so that no learned 3D backbone is required at inference. Stage 1 fine-tunes the vision encoder on 60,000 grounding records; zero-shot baselines produce no parseable BEV outputs, confirming the vocabulary requires explicit training. Stage 2 fine-tunes the full model (52M parameters, 2.4% of total) on 57,696 chain-of-thought records generated by Cosmos-Reason2-8B as teacher, spanning eight driving question types. On 2,304 held-out nuScenes frames evaluated by Gemma 4 (31B) calibrated against human review, MoRAL wins seven of eight question types over a zero-shot 8B baseline despite using four times fewer parameters, with the largest margins on question types requiring structured multi-step physics reasoning. Emergency braking recall improves from 10.8% to 47.8%, output degeneration falls from 94.1% to 20.8%, and the full pipeline fits a consumer 8 GB GPU at 42 tok/s without quantization. These results establish a reproducible foundation for compact, physics-grounded VLM reasoning on mobile edge platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。