arXiv:2603.11320cs.CV2026-03被引 1

提出统一视觉语言模型的令牌压缩方法,显著降低计算开销。

UniCompress: Token Compression for Unified Vision-Language Understanding and Generation

  • 用可学习全局元令牌引导插件式压缩与解压机制
  • 图像令牌减少4倍,推理延迟和训练成本大幅下降
  • 无需重新训练,适合资源受限的多模态应用

统一模型通过将图像编码为离散令牌,并在单个自回归框架中与文本共同处理,实现理解与生成的统一。该设计虽具架构简洁与跨模态协同优势,但大量视觉令牌带来显著计算与内存开销,制约其在如具身智能等资源受限场景的部署。本文提出统一令牌压缩算法UniCompress,大幅减少视觉令牌数量的同时保持理解与生成性能。方法引入可学习全局元令牌引导的插件式压缩与解压机制,轻量且模块化,可无缝集成至现有模型而无需全量重训练。实验表明,该方法使图像令牌减少最多4倍,显著降低推理延迟与训练成本,仅造成轻微性能损失,验证了令牌高效统一建模在真实多模态应用中的潜力。

原文摘要 · Abstract (English)

Unified models aim to support both understanding and generation by encoding images into discrete tokens and processing them alongside text within a single autoregressive framework. This unified design offers architectural simplicity and cross-modal synergy, which facilitates shared parameterization, consistent training objectives, and seamless transfer between modalities. However, the large number of visual tokens required by such models introduces substantial computation and memory overhead, and this inefficiency directly hinders deployment in resource constrained scenarios such as embodied AI systems. In this work, we propose a unified token compression algorithm UniCompress that significantly reduces visual token count while preserving performance on both image understanding and generation tasks. Our method introduces a plug-in compression and decompression mechanism guided with learnable global meta tokens. The framework is lightweight and modular, enabling efficient integration into existing models without full retraining. Experimental results show that our approach reduces image tokens by up to 4 times, achieves substantial gains in inference latency and training cost, and incurs only minimal performance degradation, which demonstrates the promise of token-efficient unified modeling for real world multimodal applications.

多模态令牌压缩统一模型效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。