通过自适应融合与双阶段剪枝，高效压缩多模态大模型视觉令牌。
CLASP: Class-Adaptive Layer Fusion and Dual-Stage Pruning for Multimodal Large Language Models
- 按类别自适应融合多层视觉特征，提升表征质量。
- 双阶段剪枝：关注重要性与覆盖率，实现精准降冗。
- 适用于多种指令与模型架构，剪枝后性能更稳定。
多模态大语言模型（MLLMs）因视觉标记序列冗余导致巨大计算开销。现有方法通常依赖单层视觉变压器（ViT）特征和静态剪枝策略，但固定配置在不同指令下易失效。为此，我们提出CLASP，一种即插即用的令牌压缩框架，基于类别自适应层融合与双阶段剪枝。首先，通过多层视觉特征融合构建类别特定的视觉表示；其次，执行双阶段剪枝：在注意力显著的枢纽标记间分配预算以保持相关性，在冗余感知的补全标记间分配预算以保证覆盖度。通过类别自适应剪枝，CLASP实现提示条件下的特征融合与预算分配，支持激进且鲁棒的视觉令牌缩减。大量实验表明，CLASP在多种基准、剪枝率及MLLM架构下均优于现有方法。代码将公开于https://github.com/Yunkaidang/CLASP。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) suffer from substantial computational overhead due to the high redundancy in visual token sequences. Existing approaches typically address this issue using single-layer Vision Transformer (ViT) features and static pruning strategies. However, such fixed configurations are often brittle under diverse instructions. To overcome these limitations, we propose CLASP, a plug-and-play token reduction framework based on class-adaptive layer fusion and dual-stage pruning. Specifically, CLASP first constructs category-specific visual representations through multi-layer vision feature fusion. It then performs dual-stage pruning, allocating the token budget between attention-salient pivot tokens for relevance and redundancy-aware completion tokens for coverage. Through class-adaptive pruning, CLASP enables prompt-conditioned feature fusion and budget allocation, allowing aggressive yet robust visual token reduction. Extensive experiments demonstrate that CLASP consistently outperforms existing methods across a wide range of benchmarks, pruning ratios, and MLLM architectures. Code will be available at https://github.com/Yunkaidang/CLASP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。