通过自适应剪枝与缓存压缩，提升多模态模型在边缘设备的部署效率。
Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression
- 动态调整各层剪枝率和键值缓存量化位宽，以性能为优化目标。
- 在LLaVA-1.5 7B/13B上实现比SparseGPT等方法更优的压缩效果。
- 无需微调即可高效压缩，适合资源受限场景的多模态模型部署。
大型多模态模型（LMMs）通过整合视觉编码器与大规模语言模型，显著提升了推理能力。然而，将LMMs压缩以部署于边缘设备仍是关键挑战。本文提出一种基于树状帕尔策估计器（Tree-structured Parzen Estimator）的自适应搜索算法，动态优化各层的剪枝率与键值（KV）缓存量化位宽，以模型性能为优化目标。该方法首次将剪枝与键值缓存量化相结合，并引入快速剪枝技术，无需额外微调或权重调整即可实现高效压缩，且不损失精度。在包括LLaVA-1.5 7B和13B在内的基准数据集上的全面评估表明，本方法在多种压缩水平下均优于SparseGPT、Wanda等先进方案。尤为突出的是，框架自动分配KV缓存压缩资源，为LMM优化树立了新标准，在大幅降低内存占用的同时保持高性能。
原文摘要 · Abstract (English)
Large multimodal models (LMMs) have advanced significantly by integrating visual encoders with extensive language models, enabling robust reasoning capabilities. However, compressing LMMs for deployment on edge devices remains a critical challenge. In this work, we propose an adaptive search algorithm that optimizes sparsity and KV cache compression to enhance LMM efficiency. Utilizing the Tree-structured Parzen Estimator, our method dynamically adjusts pruning ratios and KV cache quantization bandwidth across different LMM layers, using model performance as the optimization objective. This approach uniquely combines pruning with key-value cache quantization and incorporates a fast pruning technique that eliminates the need for additional fine-tuning or weight adjustments, achieving efficient compression without compromising accuracy. Comprehensive evaluations on benchmark datasets, including LLaVA-1.5 7B and 13B, demonstrate our method superiority over state-of-the-art techniques such as SparseGPT and Wanda across various compression levels. Notably, our framework automatic allocation of KV cache compression resources sets a new standard in LMM optimization, delivering memory efficiency without sacrificing much performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。