arXiv:2601.13974cs.CV2026-01

提出新指标STEC,评估视频采样帧的时空覆盖质量。

STEC: A Reference-Free Spatio-Temporal Entropy Coverage Metric for Evaluating Sampled Video Frames

  • 基于时空熵构建无参考评估指标,衡量采样帧的代表性与冗余度。
  • 在MSR-VTT数据集上区分随机、均匀和内容感知采样策略效果。
  • 适合用于高效视频理解中分析采样行为,不预测下游任务准确率。

视频采样是视频理解与视频-语言模型流程中的基础环节,但评估采样帧质量仍具挑战。现有指标多关注视觉质量或重建保真度,无法有效评估采样帧是否涵盖有信息量且具代表性的视频内容。本文提出时空熵覆盖率(STEC),一种简单、无参考的视频帧采样评估指标。STEC基于时空帧熵(STFE),通过熵度量每帧的空间信息强度,并结合时间分布与非冗余性评估采样效果。通过联合建模空间信息强度、时间分散性与非冗余性,STEC提供了一种轻量且合理的采样质量评估方法。在MSR-VTT测试集1k样本上的实验表明,STEC能清晰区分随机、均匀及内容感知等常见采样策略。此外,它揭示了个体视频中采样表现的鲁棒性模式,这些模式无法仅通过平均性能捕捉,凸显其作为通用评估工具的实际价值。需强调的是,STEC并非用于预测下游任务准确率,而是为受限预算下的采样行为提供任务无关的诊断信号。

原文摘要 · Abstract (English)

Frame sampling is a fundamental component in video understanding and video--language model pipelines, yet evaluating the quality of sampled frames remains challenging. Existing evaluation metrics primarily focus on perceptual quality or reconstruction fidelity, and are not designed to assess whether a set of sampled frames adequately captures informative and representative video content. We propose Spatio-Temporal Entropy Coverage (STEC), a simple and non-reference metric for evaluating the effectiveness of video frame sampling. STEC builds upon Spatio-Temporal Frame Entropy (STFE), which measures per-frame spatial information via entropy-based structural complexity, and evaluates sampled frames based on their temporal coverage and redundancy. By jointly modeling spatial information strength, temporal dispersion, and non-redundancy, STEC provides a principled and lightweight measure of sampling quality. Experiments on the MSR-VTT test-1k benchmark demonstrate that STEC clearly differentiates common sampling strategies, including random, uniform, and content-aware methods. We further show that STEC reveals robustness patterns across individual videos that are not captured by average performance alone, highlighting its practical value as a general-purpose evaluation tool for efficient video understanding. We emphasize that STEC is not designed to predict downstream task accuracy, but to provide a task-agnostic diagnostic signal for analyzing frame sampling behavior under constrained budgets.

视频采样评估指标时空分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。