用双摄像头同步拍摄聚焦与模糊视频,更好区分摩尔纹和真实纹理。
Video Demoireing using Focused-Defocused Dual-Camera System
- 通过聚焦/模糊双路视频对比,辅助识别摩尔纹
- 在多个数据集上优于现有方法,有效保留色彩一致性
- 适合图像处理、影视后期等需要精准去摩尔纹的场景
摩尔纹是数字相机采样与场景高频内容干涉产生的不必要彩色伪影。现有单摄像头去摩尔纹方法面临两大挑战:难以区分摩尔纹与视觉相似的真实纹理,且去除过程中易破坏色调一致性和时间连贯性。为此,本文提出一种双摄像头框架,同步拍摄同一场景的聚焦视频(保留高质量纹理但可能含摩尔纹)与模糊视频(摩尔纹显著减弱但纹理模糊)。利用模糊视频作为参考,帮助区分摩尔纹与真实纹理,指导聚焦视频的去摩尔纹处理。提出逐帧去摩尔纹流程:先基于光流对齐两路视频帧以解决位移和遮挡差异;再使用多尺度卷积神经网络(CNN)结合多维训练损失,以对齐后的模糊帧为引导进行去摩尔纹;最后通过联合双边滤波器,以CNN输出结果为引导,对输入聚焦帧进行滤波,保持色调与时间一致性。实验表明,本方法在多个数据集上显著优于现有最先进图像与视频去摩尔纹方法。
原文摘要 · Abstract (English)
Moire patterns, unwanted color artifacts in images and videos, arise from the interference between spatially high-frequency scene contents and the spatial discrete sampling of digital cameras. Existing demoireing methods primarily rely on single-camera image/video processing, which faces two critical challenges: 1) distinguishing moire patterns from visually similar real textures, and 2) preserving tonal consistency and temporal coherence while removing moire artifacts. To address these issues, we propose a dual-camera framework that captures synchronized videos of the same scene: one in focus (retaining high-quality textures but may exhibit moire patterns) and one defocused (with significantly reduced moire patterns but blurred textures). We use the defocused video to help distinguish moire patterns from real texture, so as to guide the demoireing of the focused video. We propose a frame-wise demoireing pipeline, which begins with an optical flow based alignment step to address any discrepancies in displacement and occlusion between the focused and defocused frames. Then, we leverage the aligned defocused frame to guide the demoireing of the focused frame using a multi-scale CNN and a multi-dimensional training loss. To maintain tonal and temporal consistency, our final step involves a joint bilateral filter to leverage the demoireing result from the CNN as the guide to filter the input focused frame to obtain the final output. Experimental results demonstrate that our proposed framework largely outperforms state-of-the-art image and video demoireing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。