针对多模态大模型4比特量化难题，提出感知模态差异的量化方法
MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models
- 按模态差异设计偏差补偿机制，保护长尾异常值
- 在ScienceQA上达76.63%准确率，优于现有4比特方案
- 适合追求高精度低比特部署的多模态模型开发者
传统后训练量化（PTQ）方法在4比特多模态大语言模型（OLLMs）上表现不佳，源于不同模态间分布异质性强烈且异常值模式各异。为此，我们提出MorphoQuant，一种感知模态的PTQ框架，旨在保留跨模态结构特征并减轻异常值损失。具体提出分布感知偏差补偿（DABC），选择性将长尾异常值融入通道级偏置；该机制在保持密集内点高精度离散化的同时，保护异常值幅度。同时提出形态导向量化函数优化（MDQFO），联合优化量化网格与偏置掩码，实现模态间细粒度对齐。在Qwen2.5-Omni模型上，于MMMU和Video-MME等基准测试中验证了优越性。尤其在ScienceQA上，我们的W4A4模型达到76.63%准确率，显著超越当前最优的W4A4方法，并意外超过全精度的W4A16基线，充分展示该框架在精度-效率权衡上的卓越表现。
原文摘要 · Abstract (English)
Conventional Post-Training Quantization (PTQ) methods struggle with 4-bit Omni-modal Large Language Models (OLLMs) due to the extreme distribution heterogeneity and disparate outlier patterns across modalities. To address this, we propose MorphoQuant, a modality-aware PTQ framework engineered to preserve cross-modal morphology and mitigate outlier loss. Specifically, we introduce Distribution-Aware Bias Compensation (DABC), which selectively absorbs long-tailed outliers into channel-wise biases. This mechanism safeguards outlier magnitudes while maintaining high-precision discretization for dense inliers, thereby preserving accurate discretization across diverse modal distribution. Complementing this, we propose Morphology-Directed Quantization Function Optimization (MDQFO) to co-optimize the quantization grid with the bias mask, ensuring fine-grained alignment across modalities. Extensive evaluations on Qwen2.5-Omni across benchmarks like MMMU and Video-MME demonstrate our approach's superiority. Notably, our W4A4 model achieves 76.63% on ScienceQA, significantly outperforming SOTA W4A4 methods and surprisingly surpassing the W4A16 baseline, which fully demonstrates the exceptional accuracy-efficiency trade-off of our framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。