arXiv:2607.09080cs.CV2026-07

用几何感知方法压缩视频令牌,提升大模型效率与可解释性。

GeoTrace: Geometry-Aware Trajectory Token Compression for Video Large Language Models

  • 分离出骨架令牌与可追踪事件令牌,避免局部过拟合。
  • 仅保留10%令牌,实现12.99倍计算量降低,性能损失不足1%。
  • 无需训练,适配多种视频大模型,适合追求高效推理的开发者。

尽管视频大语言模型在视频理解上表现优异,但其效率受限于海量视觉令牌。现有压缩方法多依赖帧级显著性或启发式合并,易聚焦局部显著区域并产生模糊融合特征。为此,我们提出GeoTrace——一种无需训练的时空令牌压缩框架,将视频证据分解为精确的骨架令牌与可追踪的残差事件令牌。具体而言,上下文最远点锚定(CFPA)保留显著、上下文一致且覆盖广泛的骨架令牌;轨迹约束残差压缩(TCRC)通过一对一时间轨迹与近流形约束压缩残差令牌,生成歧义更小的可追踪事件令牌。我们在四种视频大模型和四个基准上评估GeoTrace,结果表明其在不同架构与场景下均具有效性与泛化能力。在LLaVA-OneVision上,仅保留10%视觉令牌,即实现12.99倍的TFLOPs降低,同时保持99.1%的原始性能。总体而言,GeoTrace为高效且鲁棒的视频大模型推理提供了紧凑且可追踪的令牌表示。代码已开源。

原文摘要 · Abstract (English)

Although Video Large Language Models (Video LLMs) have shown strong performance in video understanding, their efficiency is still limited by the large number of visual tokens. Existing video token compression methods typically rely on frame-wise saliency or heuristic token merging, which can over-focus on locally salient regions and produce ambiguous fused features. To address these issues, we propose GeoTrace, a training-free spatiotemporal token compression framework that decomposes video evidence into exact skeleton tokens and traceable residual event tokens. Specifically, Contextual Farthest-Point Anchoring (CFPA) preserves salient, context-consistent, and high-coverage skeleton tokens, while Trajectory-Constrained Residual Condensation (TCRC) compresses residual tokens through one-to-one temporal trajectories and constrained near-manifold condensation, producing traceable event tokens with reduced ambiguity. We evaluate GeoTrace on four Video LLMs across four video understanding benchmarks, and the results demonstrate its effectiveness and generalization across different model architectures and scenarios. On LLaVA-OneVision, with only 10\% visual tokens retained, GeoTrace achieves a \(12.99\times\) TFLOPs reduction while preserving 99.1\% of the vanilla performance. Overall, GeoTrace offers a compact and traceable token representation for efficient and robust Video LLM inference. Code is available at \href{https://github.com/guohuan-xie/GeoTrace.git}{\texttt{Code}}.

视频理解令牌压缩大模型效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。