无需干净视频，通过运动差异自动去噪低信噪比视频
SAVeD: Learning to Denoise Low-SNR Video for Improved Downstream Performance
- 利用前景与背景运动差异，增强运动物体信号并抑制噪声
- 在多个下游任务中达到当前最优性能，且训练资源更少
- 适合无配对清晰视频的水下、医学等传感器视频处理
低信噪比视频（如水下声呐、超声波和显微镜影像）给计算机视觉模型带来挑战，尤其当缺乏配对清晰影像时。我们提出 SAVeD：一种仅使用原始噪声数据的自监督去噪方法。通过识别前景与背景运动差异，并放大强运动信号，SAVeD 提升了前景可见性，同时抑制背景与相机噪声，无需清洁视频。该方法具备一系列架构优化，使吞吐量、训练和推理速度优于现有深度学习方法。我们还引入新度量 FBD，用于检测数据集的前景-背景差异，无需清洁图像。实验表明，SAVeD 在分类、检测、跟踪和计数任务中均达领先水平，且所需训练资源更少。
原文摘要 · Abstract (English)
Low signal-to-noise ratio videos -- such as those from underwater sonar, ultrasound, and microscopy -- pose significant challenges for computer vision models, particularly when paired clean imagery is unavailable. We present Spatiotemporal Augmentations and denoising in Video for Downstream Tasks (SAVeD), a novel self-supervised method that denoises low-SNR sensor videos using only raw noisy data. By leveraging distinctions between foreground and background motion and exaggerating objects with stronger motion signal, SAVeD enhances foreground object visibility and reduces background and camera noise without requiring clean video. SAVeD has a set of architectural optimizations that lead to faster throughput, training, and inference than existing deep learning methods. We also introduce a new denoising metric, FBD, which indicates foreground-background divergence for detection datasets without requiring clean imagery. Our approach achieves state-of-the-art results for classification, detection, tracking, and counting tasks, and it does so with fewer training resource requirements than existing deep-learning-based denoising methods. Project page: https://suzanne-stathatos.github.io/SAVeD Code page: https://github.com/suzanne-stathatos/SAVeD
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。