arXiv:2510.22936cs.CV2025-10被引 2

提出PPE编码器,在压缩视觉令牌时保留空间时间结构。

PPE: Positional Preservation Embedding for Token Compression in Multimodal Large Language Models

  • 通过分离3D位置编码,让压缩后令牌包含多个原始位置信息。
  • 在多个基准上提升2%~5%,尤其在布局和时序理解任务中效果显著。
  • 无需参数、可无缝接入现有方法,适合追求高效多模态模型的开发者。

多模态大语言模型在视觉-语言任务上表现强劲,但常因冗余视觉令牌导致效率低下。现有令牌合并方法虽缩短序列长度,却忽视位置关系,破坏空间布局与时间连续性。本文提出新型编码算子——位置保留嵌入(PPE),核心在于压缩过程中保持时空结构。PPE在令牌维度显式解耦3D位置编码,使每个压缩令牌可封装来自多个原始令牌的不同位置信息。此外,PPE支持级联聚类——一种渐进式令牌压缩策略,有效提升性能保留率。该方法无参数、通用性强,可无缝集成至现有合并框架,无需调整。应用于先进令牌合并框架后,在MMBench(通用视觉理解)、TextVQA(布局理解)和VideoMME(时序理解)等多个视觉-语言基准上均实现2%~5%的一致提升,验证了保留位置线索对高效多模态推理的关键作用。代码已开源。

原文摘要 · Abstract (English)

Multimodal large language models (MLLMs) have achieved strong performance on vision-language tasks, yet often suffer from inefficiencies due to redundant visual tokens. Existing token merging methods reduce sequence length but frequently disrupt spatial layouts and temporal continuity by disregarding positional relationships. In this work, we propose a novel encoding operator dubbed as \textbf{P}ositional \textbf{P}reservation \textbf{E}mbedding (\textbf{PPE}), which has the main hallmark of preservation of spatiotemporal structure during visual token compression. PPE explicitly introduces the disentangled encoding of 3D positions in the token dimension, enabling each compressed token to encapsulate different positions from multiple original tokens. Furthermore, we show that PPE can effectively support cascade clustering -- a progressive token compression strategy that leads to better performance retention. PPE is a parameter-free and generic operator that can be seamlessly integrated into existing token merging methods without any adjustments. Applied to state-of-the-art token merging framework, PPE achieves consistent improvements of $2\%\sim5\%$ across multiple vision-language benchmarks, including MMBench (general vision understanding), TextVQA (layout understanding) and VideoMME (temporal understanding). These results demonstrate that preserving positional cues is critical for efficient and effective MLLM reasoning. Our code is available at https://github.com/MouxiaoHuang/PPE.

多模态令牌压缩位置保留LLM优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。