arXiv:2605.10050cs.CV2026-05被引 1

通过识别视频冗余为时间回声,提升长视频理解效率

EchoPrune: Interpreting Redundancy as Temporal Echoes for Efficient VideoLLMs

论文配图:EchoPrune: Interpreting Redundancy as Temporal Echoes for Efficient VideoLLMs
图 1 · 摘自论文原文
  • 将冗余帧视为可预测的时间回声,动态筛选关键视觉标记
  • 在相同令牌预算下处理20倍视频帧数,性能提升8.6%
  • 无需训练、轻量高效,适合需要高时序分辨率的视频模型

长视频理解对视频大语言模型(VideoLLMs)仍是挑战:密集采样引入大量视觉标记,稀疏采样又可能遗漏关键时序信息导致模型幻觉。现有无训练令牌压缩方法或等同处理视频为静态图像,或依赖片段级合并启发式规则,削弱细粒度时空建模并增加额外开销。本文提出EchoPrune,一种轻量级、无需训练的令牌剪枝方法,在固定大语言模型侧视觉令牌预算下提升时间分辨率。核心思想是将冗余视频令牌视为时间回声:若某令牌能从前一帧良好重建,则为时间冗余;否则可能包含新事件、运动或与查询相关的视觉证据。基于此,EchoPrune通过(i)查询引导的跨模态相关性与(ii)跨连续帧的对应匹配和回声匹配计算的时序重建误差,对视觉令牌进行评分。所选令牌保留任务相关线索与时间新颖性,同时抑制可预测冗余,使VideoLLMs能在不增加解码预算的情况下观测更多帧。在LLaVA-OV、Qwen2.5VL和Qwen3VL上六项视频理解基准的实验表明,EchoPrune使VideoLLMs在相同令牌预算下最多处理20倍视频帧数,对Qwen2.5VL-7B实现性能提升8.6%及预填充阶段5.6倍加速。

原文摘要 · Abstract (English)

Long-form video understanding remains challenging for Video Large Language Models (VideoLLMs), as the dense frame sampling introduces massive visual tokens while sparse sampling risks missing critical temporal evidence and leading to LLM hallucination. Existing training-free token reduction methods either treat videos equally as static images or rely on segment-level merging heuristics, which weaken fine-grained spatiotemporal modeling and introduce additional overhead. In this paper, we propose EchoPrune, a lightweight and training-free token pruning method that improves temporal resolution under a fixed LLM-side visual token budget. Our core idea is to interpret redundant video tokens as temporal echoes: if a token is well reconstructed from the previous frame, it is merely a temporally redundant echo; otherwise, it may capture new events, motion, or query-relevant visual evidence. Based on this insight, EchoPrune scores visual tokens by (i) query-guided crossmodal relevance and (ii) temporal reconstruction error, measured by correspondence matching and echo matching across consecutive frames. The selected tokens preserve task-relevant cues and temporal novelty while suppressing predictable redundancy, allowing VideoLLMs to observe more frames without increasing the decoding budget. Extensive experiments on LLaVA-OV, Qwen2.5VL, and Qwen3VL across six video understanding benchmarks show that EchoPrune enables VideoLLMs to process up to 20x frames under the same token budget, yielding improved performance (+8.6%) and inference speedup (5.6x for prefilling) on Qwen2.5VL-7B.

视频理解令牌剪枝时序建模高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。