动态压缩视频令牌，加速大模型推理且不损失性能
DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models
- 按解码阶段动态合并冗余帧间令牌，减少时间冗余
- 每步保留关键空间令牌，实现1.5倍加速与1.4倍内存降低
- 无需训练，可直接插入现有视频大模型中使用
视频大语言模型（VLLM）在处理复杂视频内容方面取得了显著进展，但其推理效率受限于视频输入生成的数千个视觉令牌带来的高计算成本。我们发现，与单图像输入不同，VLLM在不同解码步骤中会关注来自不同帧的视觉令牌，因此一次性剪枝策略容易误删重要令牌。为此，我们提出DyCoke，一种无需训练的令牌压缩方法，以优化令牌表示并加速VLLM。DyCoke引入即插即用的时间压缩模块，通过跨帧合并冗余令牌来最小化时间冗余，并结合动态KV缓存缩减策略，选择性地修剪空间冗余令牌。它通过在每个解码步骤动态保留关键令牌，确保高质量推理。大量实验表明，DyCoke优于先前最先进方法，在无训练条件下实现1.5倍推理加速、1.4倍内存减少的同时，性能还得到提升。
原文摘要 · Abstract (English)
Video large language models (VLLMs) have significantly advanced recently in processing complex video content, yet their inference efficiency remains constrained because of the high computational cost stemming from the thousands of visual tokens generated from the video inputs. We empirically observe that, unlike single image inputs, VLLMs typically attend visual tokens from different frames at different decoding iterations, making a one-shot pruning strategy prone to removing important tokens by mistake. Motivated by this, we present DyCoke, a training-free token compression method to optimize token representation and accelerate VLLMs. DyCoke incorporates a plug-and-play temporal compression module to minimize temporal redundancy by merging redundant tokens across frames, and applies dynamic KV cache reduction to prune spatially redundant tokens selectively. It ensures high-quality inference by dynamically retaining the critical tokens at each decoding step. Extensive experimental results demonstrate that DyCoke can outperform the prior SoTA counterparts, achieving 1.5X inference speedup, 1.4X memory reduction against the baseline VLLM, while still improving the performance, with no training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。