用大模型融合车辆传感器数据,自动生成安全提醒,提升智能电网中电动车的驾驶安全性。
Multimodal Large Language Model Framework for Safe and Interpretable Grid-Integrated EVs
- 融合视觉、定位和车端数据,通过大模型生成自然语言预警
- 真实城市道路测试验证,能准确识别行人、非机动车等危险场景
- 适合智能交通与电网协同管理,助力电动车队调度与能源规划
电动汽车融入智能电网为交通与能源系统带来新机遇,但确保驾驶员、车辆与环境间安全可解释的交互仍是关键挑战。本文提出一种基于多模态大语言模型(LLM)的框架,处理包括目标检测、语义分割及车载总线(CAN bus)遥测在内的多源传感器数据,并生成自然语言警示信息。该框架基于真实城市道路中改装车辆采集的数据进行验证,结合视觉感知(YOLOv8)、地理编码定位与车端数据,将原始传感信息转化为驾驶员可理解的内容,提升城市驾驶中的安全决策能力。案例研究显示,系统在靠近行人、骑车人及其他车辆等关键场景下能有效生成上下文感知的警报。论文强调了大语言模型在电动出行中的辅助潜力,可支持规模化车队协调、电动车负荷预测及交通驱动的能源规划,推动交通系统与电力网络协同发展。
原文摘要 · Abstract (English)
The integration of electric vehicles (EVs) into smart grids presents unique opportunities to enhance both transportation systems and energy networks. However, ensuring safe and interpretable interactions between drivers, vehicles, and the surrounding environment remains a critical challenge. This paper presents a multi-modal large language model (LLM)-based framework to process multimodal sensor data - such as object detection, semantic segmentation, and vehicular telemetry - and generate natural-language alerts for drivers. The framework is validated using real-world data collected from instrumented vehicles driving on urban roads, ensuring its applicability to real-world scenarios. By combining visual perception (YOLOv8), geocoded positioning, and CAN bus telemetry, the framework bridges raw sensor data and driver comprehension, enabling safer and more informed decision-making in urban driving scenarios. Case studies using real data demonstrate the framework's effectiveness in generating context-aware alerts for critical situations, such as proximity to pedestrians, cyclists, and other vehicles. This paper highlights the potential of LLMs as assistive tools in e-mobility, benefiting both transportation systems and electric networks by enabling scalable fleet coordination, EV load forecasting, and traffic-aware energy planning. Index Terms - Electric vehicles, visual perception, large language models, YOLOv8, semantic segmentation, CAN bus, prompt engineering, smart grid.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。