arXiv:2605.11803cs.CVcs.AI2026-05

用最优传输动态压缩视频令牌,高效保留关键信息。

OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models

论文配图:OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models
图 1 · 摘自论文原文
  • 基于最优传输构建帧间令牌压缩框架,保护重要语义内容。
  • 仅保留10%令牌,仍保持95.8%的问答性能和73.9%的时间定位性能。
  • 适合追求推理效率的视频大模型应用,无需训练即可部署。

随着视频大语言模型处理更长、更复杂的视频,跨帧累积的视觉令牌数量导致推理成本急剧上升。无需训练的令牌压缩成为缓解这一瓶颈的有效方案。然而,现有时序压缩方法主要依赖帧间令牌相似性或分割启发式规则,忽视了每个令牌在其帧内的语义角色,并未能根据每对帧的可压缩性自适应调整压缩强度。本文提出 OTT-Vid,一种基于最优传输的时序令牌压缩分配框架。该方法包含两个阶段:空间剪枝识别每帧中的代表性内容,随后在相邻帧间求解最优传输(OT),以估计时序可压缩性。我们通过非均匀令牌质量建模,保护语义重要令牌免受过度压缩,并引入兼顾特征与空间差异的局域性代价。最终的传输计划联合平衡令牌重要性与匹配成本,其总代价定义了每对帧的传输难度,用于动态分配压缩预算。在六个涵盖视频问答与时间定位任务的基准上实验表明,OTT-Vid仅保留10%的令牌,却能保持95.8%的视频问答性能和73.9%的时间定位性能,显著优于现有最先进无训练压缩方法。

原文摘要 · Abstract (English)

As Video Large Language Models (Video-LLMs) scale to longer and more complex videos, their inference cost grows rapidly due to the large volume of visual tokens accumulated across frames. Training-free token compression has emerged as a practical solution to this bottleneck. However, existing temporal compression methods rely primarily on cross-frame token similarity or segmentation heuristics, overlooking each token's semantic role within its frame and failing to adapt compression strength to the compressibility of each frame pair. In this work, we propose OTT-Vid, a transport-derived allocation framework for temporal token compression. Our approach consists of two stages: spatial pruning identifies representative content within each frame, and optimal transport (OT) is then solved between neighboring frames to estimate temporal compressibility. We formulate this OT with non-uniform token mass, which protects semantically important tokens from aggressive compression, and a locality-aware cost that captures both feature and spatial disparities. The resulting transport plan jointly balances token importance and matching cost, while its total cost defines the transport difficulty of each frame pair, which we use to allocate compression budgets dynamically. Experiments on six benchmarks spanning video question answering and temporal grounding show that OTT-Vid preserves 95.8% of VQA and 73.9% of VTG performance while retaining only 10% of tokens, consistently outperforming existing state-of-the-art training-free compression methods.

视频生成令牌压缩最优传输大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。