arXiv:2411.16312cs.CV2024-11被引 1

提出高效采样方法EPS,大幅减少超分辨率训练补丁数量。

EPS: Efficient Patch Sampling for Video Overfitting in Deep Super-Resolution Model Training

  • 基于DCT的时空特征筛选高信息量视频补丁
  • 采样补丁数减少75%至91.69%,保持视频质量
  • 适合资源受限下高速训练超分辨率模型

利用深度神经网络的过拟合特性,视频传输系统可提升带宽受限下的视频质量。现有方法通过传输过拟合的超分辨率(SR)模型流来重建高分辨率(HR)视频,但大量视频帧的训练带来巨大计算开销。为此,本文提出高效补丁采样方法EPS,从视频帧中识别最具价值的训练补丁。首先设计两种低复杂度的离散余弦变换(DCT)时空特征,直接衡量每个补丁的复杂度得分;通过分析这些特征的直方图分布,将所有可能补丁划分为不同簇,并从信息量最高的簇中选取训练补丁。采样数量根据视频内容自适应调整,平衡训练复杂度与效率。实验表明,该方法使训练补丁数量减少75.00%至91.69%(取决于分辨率与簇数),同时保持高质量重建,并显著提升训练效率。相较当前最优采样技术(EMT),补丁采样速度最高提升82.1倍。

原文摘要 · Abstract (English)

Leveraging the overfitting property of deep neural networks (DNNs) is trending in video delivery systems to enhance video quality within bandwidth limits. Existing approaches transmit overfitted super-resolution (SR) model streams for low-resolution (LR) bitstreams, which are used to reconstruct high-resolution (HR) videos at the decoder. Although these approaches show promising results, the huge computational costs of training a large number of video frames limit their practical applications. To overcome this challenge, we propose an efficient patch sampling method named EPS for video SR network overfitting, which identifies the most valuable training patches from video frames. To this end, we first present two low-complexity Discrete Cosine Transform (DCT)-based spatial-temporal features to measure the complexity score of each patch directly. By analyzing the histogram distribution of these features, we then categorize all possible patches into different clusters and select training patches from the cluster with the highest spatial-temporal information. The number of sampled patches is adaptive based on the video content, addressing the trade-off between training complexity and efficiency. Our method reduces the number of training patches by 75.00% to 91.69%, depending on the resolution and number of clusters, while preserving high video quality and greatly improving training efficiency. Our method speeds up patch sampling by up to 82.1x compared to the state-of-the-art patch sampling technique (EMT).

超分辨率视频压缩高效训练补丁采样

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。