arXiv:2512.14621cs.CV2025-12

将视频数据集压缩为关键帧,实现高效训练且性能更优。

Distill Video Datasets into Images

  • 用单帧捕捉视频语义,通过可微插值生成视频序列。
  • 在MiniUCF上比现有方法最高提升5.3%准确率。
  • 适合需要高效视频训练的场景,如资源受限设备。

数据集蒸馏旨在合成紧凑但信息丰富的数据集,使模型在小规模数据上训练仍能接近全量数据的性能。尽管图像数据已有良好效果,但视频数据的蒸馏因时间维度引入大量可学习参数而面临挑战,常导致性能不佳。本文指出核心问题在于时间维度带来的优化复杂性。我们发现单帧往往足以捕捉视频的判别语义,据此提出单帧视频数据集蒸馏(SFVD)框架,将视频压缩为每类高信息量的关键帧。利用可微插值将这些帧转换为视频序列,并与原始数据集匹配,同时仅更新帧本身以提升优化效率。为进一步融合时间信息,在匹配过程中通过通道重塑层结合真实采样视频。多基准测试表明,SFVD显著优于先前方法,在MiniUCF上最高提升5.3%,提供更高效的解决方案。

原文摘要 · Abstract (English)

Dataset distillation aims to synthesize compact yet informative datasets that allow models trained on them to achieve performance comparable to training on the full dataset. While this approach has shown promising results for image data, extending dataset distillation methods to video data has proven challenging and often leads to suboptimal performance. In this work, we first identify the core challenge in video set distillation as the substantial increase in learnable parameters introduced by the temporal dimension of video, which complicates optimization and hinders convergence. To address this issue, we observe that a single frame is often sufficient to capture the discriminative semantics of a video. Leveraging this insight, we propose Single-Frame Video set Distillation (SFVD), a framework that distills videos into highly informative frames for each class. Using differentiable interpolation, these frames are transformed into video sequences and matched with the original dataset, while updates are restricted to the frames themselves for improved optimization efficiency. To further incorporate temporal information, the distilled frames are combined with sampled real videos from real videos during the matching process through a channel reshaping layer. Extensive experiments on multiple benchmarks demonstrate that SFVD substantially outperforms prior methods, achieving improvements of up to 5.3% on MiniUCF, thereby offering a more effective solution.

视频蒸馏数据压缩可微插值

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。