将视觉语言模型的语义判断转化为潜空间分类器，实现自动驾驶实时风险检测。
LSRE: Latent Semantic Rule Encoding for Real-Time Semantic Risk Detection in Autonomous Driving
- 在世界模型潜空间中编码语言定义的安全规则，构建轻量级分类器。
- 实测达到与大模型相当的准确率，且推理速度达10赫兹。
- 适用于罕见但相似的语义场景，适合部署于真实自动驾驶系统。
现实世界的自动驾驶需遵循超越法律条文的复杂人类社会规则，如礼让急救车辆、响应交警手势或停靠校车等，这些对人类而言是直觉性的，却难以显式建模。尽管大型视觉语言模型（VLM）可解析此类语义，但其推理开销使其无法用于实时部署。本文提出LSRE框架，将稀疏采样的VLM判断转化为递归世界模型潜空间中的决策边界。通过在潜空间中编码语言定义的安全语义，构建轻量级分类器，实现每秒10次（10 Hz）的实时语义风险评估，无需逐帧调用VLM。在CARLA平台上的六个语义失效场景实验表明，LSRE性能接近大VLM基线，同时提供更早的危险预警并保持低延迟。该方法还展现出对少见但语义相似测试案例的良好泛化能力，表明语言引导的潜空间分类是自动驾驶语义安全监控的有效且可部署方案。
原文摘要 · Abstract (English)
Real-world autonomous driving must adhere to complex human social rules that extend beyond legally codified traffic regulations. Many of these semantic constraints, such as yielding to emergency vehicles, complying with traffic officers' gestures, or stopping for school buses, are intuitive for humans yet difficult to encode explicitly. Although large vision-language models (VLMs) can interpret such semantics, their inference cost makes them impractical for real-time deployment. This work proposes LSRE, a Latent Semantic Rule Encoding framework that converts sparsely sampled VLM judgments into decision boundaries within the latent space of a recurrent world model. By encoding language-defined safety semantics into a lightweight latent classifier, LSRE enables real-time semantic risk assessment at 10 Hz without per-frame VLM queries. Experiments on six semantic-failure scenarios in CARLA demonstrate that LSRE attains semantic risk detection accuracy comparable to a large VLM baseline, while providing substantially earlier hazard anticipation and maintaining low computational latency. LSRE further generalizes to rarely seen semantic-similar test cases, indicating that language-guided latent classification offers an effective and deployable mechanism for semantic safety monitoring in autonomous driving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。