用最优传输对齐视觉标记分布，高效剪枝不丢语义。
OTPrune: Distribution-Aligned Visual Token Pruning via Optimal Transport
- 基于最优传输构建分布对齐剪枝框架，无需训练。
- 在多个基准上实现更高性能与推理效率平衡。
- 理论证明单调性与次模性，适合追求稳定剪枝的开发者。
多模态大语言模型（MLLMs）具备强大的视觉-语言推理能力，但因冗余视觉标记导致高推理开销。现有剪枝方法忽视视觉表征的底层分布结构。我们提出OTPrune，一种无训练框架，将剪枝建模为通过最优传输（OT）实现分布对齐。通过最小化完整与剪枝后标记分布间的2-Wasserstein距离，OTPrune在降低推理成本的同时，保留局部多样性与全局代表性。我们推导出可计算的次模目标函数，支持高效优化，并理论证明其单调性与次模性，为稳定高效的剪枝提供原则性基础。进一步分析揭示分布对齐如何促进稳定且语义忠实的剪枝。在更广泛基准上的全面实验表明，相比最先进方法，OTPrune在性能-效率权衡上表现更优。代码已开源：https://github.com/xiwenc1/OTPrune。
原文摘要 · Abstract (English)
Multi-modal large language models (MLLMs) achieve strong visual-language reasoning but suffer from high inference cost due to redundant visual tokens. Recent work explores visual token pruning to accelerate inference, while existing pruning methods overlook the underlying distributional structure of visual representations. We propose OTPrune, a training-free framework that formulates pruning as distribution alignment via optimal transport (OT). By minimizing the 2-Wasserstein distance between the full and pruned token distributions, OTPrune preserves both local diversity and global representativeness while reducing inference cost. Moreover, we derive a tractable submodular objective that enables efficient optimization, and theoretically prove its monotonicity and submodularity, providing a principled foundation for stable and efficient pruning. We further provide a comprehensive analysis that explains how distributional alignment contributes to stable and semantically faithful pruning. Comprehensive experiments on wider benchmarks demonstrate that OTPrune achieves superior performance-efficiency tradeoffs compared to state-of-the-art methods. The code is available at https://github.com/xiwenc1/OTPrune.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。