arXiv:2604.16502cs.CV2026-04被引 1

用拓扑结构识别关键层,高效压缩视觉语言模型

Topology-Aware Layer Pruning for Large Vision-Language Models

论文配图:Topology-Aware Layer Pruning for Large Vision-Language Models
图 1 · 摘自论文原文
  • 将每层隐藏状态看作点云,用单纯复形建模其演化
  • 通过齐扎克持续同调量化层间拓扑一致性,精准保留过渡层
  • 在多个基准上优于现有方法,适合资源受限部署

大语言模型在自然语言理解与推理方面表现强劲,近年来融合视觉输入的扩展使其具备多模态处理能力。然而,大视觉语言模型(LVLMs)带来巨大的计算与内存开销,限制了其在资源受限场景中的部署。现有层剪枝方法通常依赖局部相似性度量或静态代理信号,难以捕捉表示在模型深度上的全局动态演化,常误剪关键过渡层。为此,我们提出一种面向LVLMs的拓扑感知层剪枝框架。具体而言,将各层隐藏状态表示为点云,并利用单纯复形建模其演化过程。通过齐扎克持续同调(zigzag persistent homology)量化层间拓扑一致性,实现自适应剪枝,以保留关键表征过渡。在多种多模态基准上的大量实验表明,该框架在广泛稀疏率下均持续优于现有剪枝方法。代码已开源:https://github.com/zpc456/TopoVLM。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated strong capabilities in natural language understanding and reasoning, while recent extensions that incorporate visual inputs enable them to process multimodal information. Despite these advances, Large Vision-Language Models (LVLMs) incur substantial computational and memory costs, hindering deployment in resource-constrained scenarios. Existing layer pruning methods typically rely on local similarity metrics or static proxy signals, failing to capture the global and dynamic evolution of representations across model depth, which often leads to the removal of transition-critical layers. To address this limitation, we propose a topology-aware layer pruning framework for LVLMs. Specifically, we represent layer wise hidden states as point clouds and models their evolution using \textit{simplicial complexes}. By leveraging \textit{zigzag persistent homology}, we quantify inter-layer topological consistency and enable adaptive pruning that preserves critical representational transitions. Extensive experiments on diverse multimodal benchmarks demonstrate that the proposed framework consistently outperforms existing pruning methods across a wide range of sparsity ratios. Our code is available at https://github.com/zpc456/TopoVLM.

模型剪枝视觉语言模型拓扑分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。