arXiv:2501.18545cs.CV2025-01ICCV被引 1

首个真实世界下屏摄像头视频数据集,用于修复显示面板导致的画质退化。

UDC-VIT: A Real-World Video Dataset for Under-Display Cameras

  • 搭建同步采集系统,获取同一场景的清晰与下屏摄像头退化视频对。
  • 通过傅里叶变换实现帧级对齐,确保退化模式真实可比。
  • 实验证明合成数据训练模型无效,强调真实数据对修复的关键作用。

尽管下屏摄像头(UDC)是先进的成像系统,但显示屏显著降低图像质量,引入低透光率、模糊、噪声和耀斑等问题。由于退化复杂且耀斑模式多样,解决这些问题极具挑战。然而,目前尚无包含真实世界下屏摄像头退化视频的数据集。本文提出首个真实世界下屏摄像头视频数据集 UDC-VIT。与现有数据集不同,UDC-VIT仅包含用于人脸识别的人体动作视频。我们设计了一套视频采集系统,能同步获取同一场景的干净视频与下屏摄像头退化视频。随后,利用离散傅里叶变换(DFT)进行逐帧对齐。我们将 UDC-VIT 与六个代表性下屏摄像头静态图像数据集及两个现有下屏摄像头视频数据集进行对比。使用六种深度学习模型,对比了 UDC-VIT 与一个现有合成下屏摄像头视频数据集。结果表明,在早期合成数据集上训练的模型在真实退化视频上表现不佳,无法反映实际退化特征。我们还通过人脸识别准确率评估了修复效果,关联了 PSNR、SSIM 与 LPIPS 指标。该数据集已发布于官方 GitHub 仓库。

原文摘要 · Abstract (English)

Even though an Under-Display Camera (UDC) is an advanced imaging system, the display panel significantly degrades captured images or videos, introducing low transmittance, blur, noise, and flare issues. Tackling such issues is challenging because of the complex degradation of UDCs, including diverse flare patterns. However, no dataset contains videos of real-world UDC degradation. In this paper, we propose a real-world UDC video dataset called UDC-VIT. Unlike existing datasets, UDC-VIT exclusively includes human motions for facial recognition. We propose a video-capturing system to acquire clean and UDC-degraded videos of the same scene simultaneously. Then, we align a pair of captured videos frame by frame, using discrete Fourier transform (DFT). We compare UDC-VIT with six representative UDC still image datasets and two existing UDC video datasets. Using six deep-learning models, we compare UDC-VIT and an existing synthetic UDC video dataset. The results indicate the ineffectiveness of models trained on earlier synthetic UDC video datasets, as they do not reflect the actual characteristics of UDC-degraded videos. We also demonstrate the importance of effective UDC restoration by evaluating face recognition accuracy concerning PSNR, SSIM, and LPIPS scores. UDC-VIT is available at our official GitHub repository.

视频数据集下屏摄像头图像修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。