Espresso通过分离压缩时空特征，实现高效长视频理解。
Espresso: High Compression For Rich Extraction From Videos for Your Vision-Language Model
- 分离压缩空间与时间特征，生成固定长度序列
- 固定长度编码在长视频任务上表现优于传统池化方法
- 适合需要高效处理长视频的视觉语言模型应用
视觉语言模型（VLMs）在图像与文本关联任务中表现优异，但扩展到长视频时面临令牌数量激增的挑战。现有基于时间或空间局部聚合的压缩方法虽流行，但牺牲了对流式处理和高效理解至关重要的固定长度表示优势。本文提出$ exttt{Espresso}$，一种将时空特征分别压缩为固定长度序列的新架构，兼顾高效编码与强长视频推理能力。实验表明，结合分段处理的固定长度压缩，相比池化方法更具可扩展性和竞争力。结果证明，合理设计与训练的固定长度投影器仍是视频-语言建模的可行基础。
原文摘要 · Abstract (English)
Recent advances in vision-language models (VLMs) have shown great promise in connecting images and text, but extending these models to long videos remains challenging due to the rapid growth in token counts. Models that compress videos by local aggregation in time or space have become popular for handling long-form inputs; however, these pooling-based projectors sacrifice the benefits of fixed-length representations that are crucial for streaming and efficient video understanding. We introduce $\texttt{Espresso}$, a new architecture that separately compresses spatial and temporal features into fixed-length sequences. $\texttt{Espresso}$ enables efficient video encoding while maintaining strong long-form reasoning capabilities. Experiments show that fixed-length compression combined with segment-wise processing offers a scalable and competitive alternative to pooling-based approaches. Our results demonstrate that fixed-length projectors, when properly designed and trained, remain a viable foundation for video-language modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。