通过加权奇异值分解，实现低精度视觉语言模型的高效推理。
WSVD: Weighted Low-Rank Approximation for Fast and Efficient Execution of Low-Precision Vision-Language Models
- 在更细粒度上应用加权SVD，优化计算模式。
- 实现超过1.8倍的解码速度提升，保持准确率不变。
- 适合追求推理加速的视觉语言模型部署场景。
奇异值分解（SVD）已成为降低视觉语言模型（VLMs）计算负担的重要技术，在图像字幕生成和视觉问答等任务中具有核心作用。尽管已有多种高效SVD变体用于实现低秩操作，但实际执行中仍难以显著降低延迟。为此，我们引入一种新的计算范式，将SVD应用于更细粒度的权重结构，实现了可测量的执行延迟降低。同时，考虑到权重元素的重要性差异，我们在SVD过程中自适应地分配相对重要性，以更好保留精度，并进一步结合权重与激活的量化，构建出高度高效的VLM。整体上，我们提出~ extit{加权奇异值分解}（WSVD），在保持精度的同时，实现超过1.8倍的解码速度提升。代码已开源： exttt{https://github.com/SAI-Lab-NYU/WSVD}。
原文摘要 · Abstract (English)
Singular Value Decomposition (SVD) has become an important technique for reducing the computational burden of Vision Language Models (VLMs), which play a central role in tasks such as image captioning and visual question answering. Although multiple prior works have proposed efficient SVD variants to enable low-rank operations, we find that in practice it remains difficult to achieve substantial latency reduction during model execution. To address this limitation, we introduce a new computational pattern and apply SVD at a finer granularity, enabling real and measurable improvements in execution latency. Furthermore, recognizing that weight elements differ in their relative importance, we adaptively allocate relative importance to each element during SVD process to better preserve accuracy, then extend this framework with quantization applied to both weights and activations, resulting in a highly efficient VLM. Collectively, we introduce~\textit{Weighted SVD} (WSVD), which outperforms other approaches by achieving over $1.8\times$ decoding speedup while preserving accuracy. We open source our code at: \href{https://github.com/SAI-Lab-NYU/WSVD}{\texttt{https://github.com/SAI-Lab-NYU/WSVD}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。