arXiv:2512.16349cs.CVcs.AI2025-12被引 2

边端协同推理框架,按需传输细节图像,降通信量不丢精度。

Collaborative Edge-to-Server Inference for Vision-Language Models

  • 先用缩略图全局推理,再按注意力选重点区域重传局部图。
  • 在多个VQA基准上通信量减少超60%,准确率几乎无损。
  • 适合边缘设备算力弱、带宽低的视觉语言模型部署场景。

我们提出一种面向视觉语言模型(VLMs)的边端协同推理框架,降低通信开销同时保持推理精度。传统部署中,边缘设备捕获的视觉数据需上传至服务器进行推理,但全分辨率图像传输带来高通信成本。过度压缩会丢失细粒度信息,导致精度下降。为此,我们设计了一种两阶段通信高效框架:第一阶段,服务器对缩略图进行推理,并量化输出标记的最小熵;若最小熵超过预设阈值,则利用VLM内部注意力机制识别感兴趣区域(RoI),请求边缘设备发送该区域的细节保留局部图像;第二阶段,服务器联合全局与局部图像优化推理结果。该选择性重传策略仅传输必要视觉内容。实验表明,该框架在多个VQA基准上显著降低通信开销,同时维持接近原始精度。

原文摘要 · Abstract (English)

We propose a collaborative edge-to-server inference framework for vision-language models (VLMs) that reduces communication cost while maintaining inference accuracy. In typical deployments, visual data captured at edge devices (clients) is transmitted to the server for VLM inference. However, transmitting full-resolution images incurs high communication cost. Conversely, aggressive downsizing or excessive compression to mitigate communication overhead can discard fine-grained details, leading to accuracy degradation. To overcome this limitation, we design a communication-efficient two-stage framework. In the first stage, the server performs inference on the downsized thumbnail (global image) and quantifies the min-entropy of the output tokens. If the min-entropy exceeds a predefined threshold, the server identifies a region of interest (RoI) using the VLM's internal attention and requests the edge device to send a detail-preserved local image of the RoI. The server then refines its inference by jointly leveraging the global and local images. This selective retransmission strategy ensures that only essential visual content is additionally transmitted. Experimental results consistently confirm that the proposed framework substantially reduces communication overhead while maintaining inference accuracy across diverse VQA benchmarks.

边端协同视觉语言模型通信优化注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。