让车载小模型与边缘大模型协作,按时效性智能调度,提升自动驾驶感知精度。
TALSC: Timeliness-Aware Large-Small VLM Collaboration for Infrastructure-Assisted Autonomous Driving

- 基于信息年龄(AoI)建模,动态权衡延迟与任务性能
- 在nuScenes数据集上实现12.6%的微平均F1分数提升
- 适合研究车路协同、实时视觉语言模型的开发者
自动驾驶系统中,车载算力有限,车辆只能部署小型视觉语言模型(SVLMs),感知与推理能力受限。通过车路协同,车辆可与边缘服务器的大规模视觉语言模型(LVLMs)协作,缓解资源瓶颈。但在动态交通环境中,感知数据时效性迅速下降,信息延迟成为关键问题。为此,本文提出时序感知的大小模型协同框架TALSC,首先建立VLM推理过程中的信息年龄(AoI)演化模型,揭示其与令牌长度及任务性能的耦合关系,构建通用时效性度量指标。在此基础上,设计TALSC在线调度算法,针对调度决策对未来的延迟影响及调度时未知输出令牌数的问题,采用李雅普诺夫漂移加估计惩罚机制,提供性能保障。仿真中,基于nuScenes数据集进行案例分析,拟合出时效性度量,并验证TALSC在多种通信与计算条件下均优于基线方法,相比最佳基线实现最高达12.6%的归一化微平均F1提升。
原文摘要 · Abstract (English)
The deployment of Vision-Language Models (VLMs) in autonomous driving (AD) systems is constrained by on-board computing power, restricting vehicles to small VLMs (SVLMs) with limited perception and reasoning capabilities. Infrastructure-assisted AD alleviates this resource constraint by enabling collaboration with large VLMs (LVLMs) at edge servers. However, in dynamic vehicular environments, the utility of sensory data for downstream tasks decays rapidly, making timeliness of information a critical concern. To balance the accuracy gains of LVLMs with their latency-induced timeliness degradation, we develop a Timeliness-Aware Large-Small VLM Collaboration (TALSC) framework. Specifically, we first model the Age of Information (AoI) evolution for VLM inference and characterize the coupling among AoI, token length, and task performance to formulate a general timeliness metric. Building on this, we propose the TALSC online scheduling algorithm. Since scheduling decisions have a delayed impact on future timeliness metric and the output token number is unknown at scheduling time, we design a Lyapunov drift-plus-estimated-penalty algorithm and provides a guaranteed performance. In simulation, we first conduct a case study to derive a fitted timeliness metric based on nuScenes dataset, and further show that TALSC outperforms baselines under various communication and computing settings, achieving up to a 12.6\% normalized improvement in Micro-F1 score compared with the best-performing baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。