针对多模态大模型量化难题，提出感知模态的平滑量化方法。
MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models
- 按模态分别学习平滑因子，避免跨模态信息错配
- 用SVD白化将多模态激活差异转为低秩，实现统一量化
- 在双模态与三模态模型上均表现稳定，适合高效部署多模态模型
后训练量化（PTQ）在大型语言模型中已取得显著进展，但应用于多模态大模型（MLLMs）时面临巨大挑战。本文以SmoothQuant为例分析，发现两个关键问题：平滑错位（Smoothing Misalignment）和跨模态计算不变性缺失。为此，我们提出模态感知平滑量化（MASQuant），引入两项创新：(1) 模态感知平滑（MAS），学习独立的、模态特定的平滑因子，防止平滑错位；(2) 跨模态补偿（CMC），通过SVD白化将多模态激活差异转化为低秩形式，实现跨模态统一量化。MASQuant在双模态与三模态MLLM上均展现出稳定的量化性能。实验表明，其在主流PTQ算法中具有竞争力。代码已开源：https://github.com/alibaba/EfficientAI。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) with computational invariance for Large Language Models~(LLMs) have demonstrated remarkable advances, however, their application to Multimodal Large Language Models~(MLLMs) presents substantial challenges. In this paper, we analyze SmoothQuant as a case study and identify two critical issues: Smoothing Misalignment and Cross-Modal Computational Invariance. To address these issues, we propose Modality-Aware Smoothing Quantization (MASQuant), a novel framework that introduces (1) Modality-Aware Smoothing (MAS), which learns separate, modality-specific smoothing factors to prevent Smoothing Misalignment, and (2) Cross-Modal Compensation (CMC), which addresses Cross-modal Computational Invariance by using SVD whitening to transform multi-modal activation differences into low-rank forms, enabling unified quantization across modalities. MASQuant demonstrates stable quantization performance across both dual-modal and tri-modal MLLMs. Experimental results show that MASQuant is competitive among the state-of-the-art PTQ algorithms. Source code: https://github.com/alibaba/EfficientAI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。