提出可自适应剪枝的框架，提升大模型推理效率。
Pruning All-Rounder: Rethinking and Improving Inference Efficiency for Large Vision Language Models
- 用元路由动态调度令牌和层的剪枝流程
- 无需重训练，在多个数据集上保持高精度
- 灵活适配不同加速需求，适合部署场景
尽管大视觉语言模型（LVLMs）取得了显著成果，但其高昂的计算成本阻碍了广泛应用。现有方法多分为依赖参数或依赖令牌的策略，前者需重新训练以恢复性能，后者难以稳定选择相关令牌。本文系统分析上述挑战，提出新型框架Pruning All-Rounder（PAR）。不同于以往工作，PAR设计元路由，自适应地统筹令牌与层的剪枝流。通过自监督学习，实现性能与效率的优越平衡。特别地，PAR具备高度灵活性，提供多种剪枝版本以应对多样加速场景。代码已开源。
原文摘要 · Abstract (English)
Although Large Vision-Language Models (LVLMs) have achieved impressive results, their high computational costs pose a significant barrier to wide application. To enhance inference efficiency, most existing approaches can be categorized as parameter-dependent or token-dependent strategies to reduce computational demands. However, parameter-dependent methods require retraining LVLMs to recover performance while token-dependent strategies struggle to consistently select the most relevant tokens. In this paper, we systematically analyze the above challenges and provide a series of valuable insights for inference acceleration. Based on these findings, we propose a novel framework, the Pruning All-Rounder (PAR). Different from previous works, PAR develops a meta-router to adaptively organize pruning flows across both tokens and layers. With a self-supervised learning manner, our method achieves a superior balance between performance and efficiency. Notably, PAR is highly flexible, offering multiple pruning versions to address a range of acceleration scenarios. The code for this work is publicly available at https://github.com/ASGO-MM/Pruning-All-Rounder.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。