arXiv:2605.19322cs.CV2026-05被引 1

动态分配视频令牌预算,实现高效压缩与长时序感知。

DynaTok: Temporally Adaptive and Positional Bias-Aware Token Compression for Video-LLMs

论文配图:DynaTok: Temporally Adaptive and Positional Bias-Aware Token Compression for Video-LLMs
图 1 · 摘自论文原文
  • 基于时间与空间双重自适应机制,动态分配令牌数量。
  • 90%令牌压缩下仍保持超95%基准准确率,性能领先。
  • 无需重训练,适配主流视频大模型,适合实时视频理解场景。

视频大语言模型(Video-LLMs)虽显著提升了多模态推理能力,但长视频序列产生的海量视觉令牌带来巨大计算开销,限制了实际部署。现有无训练令牌压缩方法仅依赖注意力强度作为语义重要性代理,忽略位置偏差且局限于短期时序局部性,导致时空冗余。本文提出DynaTok,一种无训练、时序自适应且具备偏差感知的令牌压缩框架,可在时空维度动态分配令牌预算。通过轻量级指数移动平均(EMA)记忆,时间预算分配(TBA)模块为重复帧减少令牌,为新颖帧增加令牌,捕捉长期时序变化;空间预算分配(SBA)模块则利用激活注意力图选择空间多样且语义重要的特征,结合空间记忆减少已选区域冗余,缓解位置偏差。DynaTok可无缝集成至LLaVA-OneVision和LLaVA-Video等模型,无需重训练,显著提升压缩效率。在四个代表性VideoQA基准(MVBench、LongVideoBench、MLVU、VideoMME)上,即使压缩率达90%,仍保持超过95%的基线准确率,优于近期无训练方法。结果表明,DynaTok为高效鲁棒的视频推理提供了可靠基础,推动未来视频大模型实现实时流式理解。

原文摘要 · Abstract (English)

Recent advances in Video Large Language Models (Video-LLMs) have greatly expanded multimodal reasoning capabilities. However, the massive number of visual tokens extracted from long video sequences incurs prohibitive computational costs, limiting their deployment in real-world scenarios. Existing training-free token compression methods select tokens based on attention magnitude as a proxy for semantic importance, but often overlook positional bias and rely only on short-term temporal locality, leading to redundant spatio-temporal coverage and inefficient token usage. We present DynaTok, a training-free, temporally adaptive and bias-aware token compression framework that allocates token budgets across both temporal and spatial dimensions. Through a lightweight exponential moving average (EMA) memory, the Temporal Budget Allocation (TBA) module dynamically assigns fewer tokens to redundant frames and more to novel frames, capturing long-term temporal variation. The Spatial Budget Allocation (SBA) module complements this by selecting spatially diverse and semantically important features using activation-based attention maps, while leveraging a spatial memory to reduce redundancy from previously selected regions and mitigate positional bias. DynaTok integrates seamlessly with existing Video-LLMs such as LLaVA-OneVision and LLaVA-Video without retraining, and effectively preserves semantic coverage under aggressive compression. Experiments on four representative VideoQA benchmarks-MVBench, LongVideoBench, MLVU, and VideoMME-show that DynaTok retains over 95% of baseline accuracy even with a 90% token reduction, surpassing recent training-free approaches. These results demonstrate that DynaTok provides a principled foundation for efficient and robust video reasoning, paving the way toward real-time streaming video understanding with future Video-LLMs.

视频理解令牌压缩大模型优化时序建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。