不训练、零延迟，用最大体积原理选视频关键帧提升理解效果
MaxInfo: A Training-Free Key-Frame Selection Method Using Maximum Volume for Enhanced Video Understanding
- 基于嵌入空间体积最大化，无须训练自动选出最具代表性的帧
- 在LongVideoBench等数据集上提升3.28%~6.4%，长视频理解更准确
- 可直接接入现有模型，适合追求高效推理的开发者使用
现代视频大语言模型通常采用均匀采样，但因帧冗余和内容差异常丢失关键信息。本文提出首个无需训练的MaxInfo方法，基于最大体积原则，在快速版、慢速版及分块版中选择最具代表性的视频帧。通过最大化所选嵌入向量形成的几何体积，确保覆盖嵌入空间中最具信息量的区域，有效减少冗余并保留多样性。该方法提升了输入表示质量，在多个基准测试中显著增强长视频理解能力：在LongVideoBench上，LLaVA-Video-7B提升3.28%，EgoSchema上提升6.4%；对LLaVA-Video-72B和MiniCPM4.5分别提升3.47%和3.44%。方法实现简单，兼容现有模型，无需额外训练，延迟极低，是传统均匀采样的高效替代方案。代码已开源。
原文摘要 · Abstract (English)
Modern Video Large Language Models (VLLMs) often rely on uniform frame sampling for video understanding, but this approach frequently fails to capture critical information due to frame redundancy and variations in video content. We propose MaxInfo, the first training-free method based on the maximum volume principle, which is available in Fast and Slow versions and a Chunk-based version that selects and retains the most representative frames from a video. By maximizing the geometric volume formed by selected embeddings, MaxInfo ensures that the chosen frames cover the most informative regions of the embedding space, effectively reducing redundancy while preserving diversity. This method enhances the quality of input representations and improves long video comprehension performance across benchmarks. For instance, MaxInfo achieves a 3.28% improvement on LongVideoBench and a 6.4% improvement on EgoSchema for LLaVA-Video-7B. Moreover, MaxInfo boosts LongVideoBench performance by 3.47% on LLaVA-Video-72B and 3.44% on MiniCPM4.5. The approach is simple to implement and works with existing VLLMs without the need for additional training and very lower latency, making it a practical and effective alternative to traditional uniform sampling methods. Our code are available at https://github.com/FusionBrainLab/MaxInfo.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。