系统研究视频数据压缩，发现不同方法在难易场景下表现各异。
A Large-Scale Study on Video Action Dataset Condensation
- 用滑动窗口处理视频时间信息，简单有效
- 数据蒸馏在复杂场景优于样本选择
- 提出统一评估标准，适合作为基准参考
近期，数据压缩在图像领域取得显著进展。与图像不同，视频具有额外的时间维度，蕴含大量冗余信息，使得压缩更为关键。然而，视频数据压缩仍属探索不足领域。本文通过大规模实验，系统设计并公平对比，深入分析三个核心问题：（1）视频数据的时间处理方式，（2）视频数据压缩的评估协议，（3）压缩算法向时空域的适应性。研究发现：（i）标注方法显著影响压缩效果，（ii）简单的滑动窗口采样对时间处理有效，（iii）数据蒸馏在困难场景中表现更优，而样本选择在简单场景中更佳。此外，我们提出统一评估协议，实现对四个主流动作识别数据集（HMDB51、UCF101、SSv2、K400）的最先进性能。代码已开源：https://github.com/MCG-NJU/Video-DC。
原文摘要 · Abstract (English)
Recently, dataset condensation has made significant progress in the image domain. Unlike images, videos possess an additional temporal dimension, which harbors considerable redundant information, making condensation even more crucial. However, video dataset condensation still remains an underexplored area. We aim to bridge this gap by providing a large-scale study with systematic design and fair comparison. Specifically, our work delves into three key aspects to provide valuable empirical insights: (1) temporal processing of video data, (2) the evaluation protocol for video dataset condensation, and (3) adaptation of condensation algorithms to the space-time domain. From this study, we derive several intriguing observations: (i) labeling methods greatly influence condensation performance, (ii) simple sliding-window sampling is effective for temporal processing, and (iii) dataset distillation methods perform better in challenging scenarios, while sample selection methods excel in easier ones. Furthermore, we propose a unified evaluation protocol for the fair comparison of different condensation algorithms and achieve state-of-the-art results on four widely-used action recognition datasets: HMDB51, UCF101, SSv2 and K400. Our code is available at https://github.com/MCG-NJU/Video-DC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。