通过帧间相似性压缩视频令牌，显著降低推理成本。
TTF: Temporal Token Fusion for Efficient Video-Language Model
- 按时间窗口搜索相似帧，自动融合冗余视觉令牌
- 减少67%令牌，保持99.5%准确率，仅增0.16 GFLOPs开销
- 无需训练、即插即用，适合长视频理解场景
视频-语言模型（VLM）在视频长度增加时面临推理成本快速上升的问题。例如，32帧、448×448分辨率的视频在Qwen3-VL中已产生超过8,000个视觉令牌，导致大语言模型（LLM）预填充成为主要吞吐瓶颈。现有方法多依赖全局相似性或注意力引导压缩，常带来性能损失。本文提出无需训练、可即插即用的时空令牌融合（Temporal Token Fusion, TTF）框架，利用视频中固有的时间冗余特性。TTF自动选择锚定帧，对后续每帧在局部窗口（如3×3）内进行相似性搜索，将超过阈值的令牌进行融合。压缩序列通过坐标重对齐保持前后位置一致性，实现与现有VLM流程无缝集成。在Qwen3-VL-8B上，阈值t=0.70时，TTF移除约67%的视觉令牌，保留99.5%的基线准确率，仅引入约0.16 GFLOPs的匹配开销。TTF为视频理解提供了高效实用的解决方案。
原文摘要 · Abstract (English)
Video-language models (VLMs) face rapid inference costs as visual token counts scale with video length. For example, 32 frames at $448{\times}448$ resolution already yield >8,000 visual tokens in Qwen3-VL, making LLM prefill the dominant throughput bottleneck. Existing methods often rely on global similarity or attention-guided compression, incurring offsets to their gains. We propose \textbf{Temporal Token Fusion (TTF)}, a training-free, plug-and-play pre-LLM token compression framework that exploits structured temporal redundancy in video. TTF automatically selects an anchor frame, then for each subsequent frame, performs a local window similarity search (e.g.,$3\times 3$), fusing tokens that exceed a threshold. The compressed sequence maintains positional consistency across both prefill and decoding through coordinate realignment, enabling seamless integration with existing VLM pipelines. On Qwen3-VL-8B with threshold t=0.70, TTF removes about 67\% of visual tokens while retaining 99.5\% of the baseline accuracy and introducing only ${\approx}0.16$\,GFLOPs of matching overhead. Overall, TTF offers a practical, efficient solution for video understanding. The code is available at \href{https://github.com/Cominder/ttf}{https://github.com/Cominder/ttf}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。