arXiv:2509.21259cs.NIcs.AI2025-09被引 10

用视觉模型压缩交通图像,云端用大模型分析,传输量减少99.9%。

Semantic Edge-Cloud Communication for Real-Time Urban Traffic Surveillance with ViT and LLMs over Mobile Networks

  • 用YOLOv11定位重点区域,ViT生成小尺寸图像嵌入
  • 传输数据量减少99.9%,云端重建图像仍能保持89%分析准确率
  • 适合部署在带宽受限的移动网络上的实时城市交通监控

实时城市交通监控对智能交通系统至关重要,可保障道路安全、优化车流、追踪车辆轨迹并预防碰撞。在城市环境中部署边缘摄像头是常规做法,但将其与智能模型结合需对动态交通场景有深度理解,并支持用户交互响应。尽管多模态大语言模型(LLMs)可解析交通图像并生成信息性回应,但因其计算开销高,难以在边缘设备上部署,因此必须将推理任务交由云端完成,导致图像数据需从边缘传至云端,受限于带宽,易引发延迟,影响实时性能。为此,我们提出一种语义通信框架,显著降低传输开销。方法包括:使用YOLOv11检测感兴趣区域(RoIs),裁剪相关图像片段,并通过视觉变换器(ViT)将其转换为紧凑的嵌入向量,再传输至云端;云端图像解码器重建裁剪图像,再交由多模态LLM生成交通状况描述。该方案实现数据传输量减少99.9%,同时在重建图像上保持89%的LLM响应准确率(原图裁剪为93%)。结果表明,基于ViT与多模态大模型的边缘-云语义通信在实时交通监控中高效且实用。

原文摘要 · Abstract (English)

Real-time urban traffic surveillance is vital for Intelligent Transportation Systems (ITS) to ensure road safety, optimize traffic flow, track vehicle trajectories, and prevent collisions in smart cities. Deploying edge cameras across urban environments is a standard practice for monitoring road conditions. However, integrating these with intelligent models requires a robust understanding of dynamic traffic scenarios and a responsive interface for user interaction. Although multimodal Large Language Models (LLMs) can interpret traffic images and generate informative responses, their deployment on edge devices is infeasible due to high computational demands. Therefore, LLM inference must occur on the cloud, necessitating visual data transmission from edge to cloud, a process hindered by limited bandwidth, leading to potential delays that compromise real-time performance. To address this challenge, we propose a semantic communication framework that significantly reduces transmission overhead. Our method involves detecting Regions of Interest (RoIs) using YOLOv11, cropping relevant image segments, and converting them into compact embedding vectors using a Vision Transformer (ViT). These embeddings are then transmitted to the cloud, where an image decoder reconstructs the cropped images. The reconstructed images are processed by a multimodal LLM to generate traffic condition descriptions. This approach achieves a 99.9% reduction in data transmission size while maintaining an LLM response accuracy of 89% for reconstructed cropped images, compared to 93% accuracy with original cropped images. Our results demonstrate the efficiency and practicality of ViT and LLM-assisted edge-cloud semantic communication for real-time traffic surveillance.

边缘计算视觉语言模型交通监控语义通信

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。