根据视频语义动态调整压缩分辨率,提升视频数据蒸馏效率
Dynamic-Aware Video Distillation: Optimizing Temporal Resolution Based on Video Semantics
- 用强化学习预测合成视频的最优时间分辨率
- 在多个视频数据集上实现显著性能提升
- 适合关注视频高效压缩与智能蒸馏的研究者
随着视觉任务和数据集、模型规模的快速扩展,视觉数据集中的冗余问题已成为研究重点。为解决此问题,数据蒸馏(DD)作为一种生成高度紧凑且冗余较少的合成数据集的方法应运而生。然而,尽管图像数据集的DD已得到广泛研究,视频数据集的DD仍鲜有探索。视频数据因包含时间信息及不同类别间冗余程度差异大而面临独特挑战。现有方法假设所有视频语义下的时间冗余一致,限制了其在视频数据上的表现。本文提出动态感知视频蒸馏(DAViD),一种基于强化学习(RL)的方法,用于预测合成视频的最优时间分辨率。设计了教师在环的奖励函数以更新RL代理策略。据我们所知,这是首个在视频数据蒸馏中引入基于视频语义自适应时间分辨率的研究。实验表明,该方法显著优于现有DD方法,在多个视频数据集上均取得显著性能提升,为未来更高效、语义自适应的视频数据蒸馏研究开辟了道路。
原文摘要 · Abstract (English)
With the rapid development of vision tasks and the scaling on datasets and models, redundancy reduction in vision datasets has become a key area of research. To address this issue, dataset distillation (DD) has emerged as a promising approach to generating highly compact synthetic datasets with significantly less redundancy while preserving essential information. However, while DD has been extensively studied for image datasets, DD on video datasets remains underexplored. Video datasets present unique challenges due to the presence of temporal information and varying levels of redundancy across different classes. Existing DD approaches assume a uniform level of temporal redundancy across all different video semantics, which limits their effectiveness on video datasets. In this work, we propose Dynamic-Aware Video Distillation (DAViD), a Reinforcement Learning (RL) approach to predict the optimal Temporal Resolution of the synthetic videos. A teacher-in-the-loop reward function is proposed to update the RL agent policy. To the best of our knowledge, this is the first study to introduce adaptive temporal resolution based on video semantics in video dataset distillation. Our approach significantly outperforms existing DD methods, demonstrating substantial improvements in performance. This work paves the way for future research on more efficient and semantic-adaptive video dataset distillation research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。