提出新型向量量化方法，实现边缘端与云端协同的视觉语言模型高效推理。
Aligned Vector Quantization for Edge-Cloud Collabrative Vision-Language Models
- 设计对齐向量量化算法，压缩中间特征以支持模型分片执行。
- 中间特征压缩率达1365倍，传输开销降低96.8%。
- 推理速度提升2-15倍，准确率仅下降2.23%以内，适合资源受限场景。
视觉语言模型（VLMs）是视觉问答（VQA）系统的核心，通常因计算需求高而部署在云端。然而，纯云端方案未充分利用边缘计算资源，且需传输原始图像，带来高额带宽开销。本文提出一种边缘-云端协同的VQA系统LLaVA-AlignedVQ，引入新型对齐向量量化算法（AlignedVQ），在不损失精度的前提下高效压缩中间特征，支持模型分片执行。实验表明，该系统实现约1365倍的中间特征压缩率，相比传输JPEG90压缩图像，数据传输开销降低96.8%；推理速度提升2-15倍，准确率在八个VQA数据集上保持在原模型性能的-2.23%至+1.6%之间，优于纯云端方案。
原文摘要 · Abstract (English)
Vision Language Models (VLMs) are central to Visual Question Answering (VQA) systems and are typically deployed in the cloud due to their high computational demands. However, this cloud-only approach underutilizes edge computational resources and requires significant bandwidth for transmitting raw images. In this paper, we introduce an edge-cloud collaborative VQA system, called LLaVA-AlignedVQ, which features a novel Aligned Vector Quantization algorithm (AlignedVQ) that efficiently compress intermediate features without compromising accuracy to support partitioned execution. Our experiments demonstrate that LLaVA-AlignedVQ achieves approximately 1365x compression rate of intermediate features, reducing data transmission overhead by 96.8% compared to transmitting JPEG90-compressed images to the cloud. LLaVA-AlignedVQ achieves an inference speedup of 2-15x while maintaining high accuracy, remaining within -2.23% to +1.6% of the original model's accuracy performance across eight VQA datasets, compared to the cloud-only solution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。