单模型同时生成与修复低质量立体视频，提升画质与一致性。
Restereo: Diffusion stereo video generation and restoration
- 用退化数据微调+扭曲掩码条件，统一处理立体生成与修复。
- 在小规模合成数据上训练，仍能有效提升低分辨率真实视频画质。
- 适合需要兼顾画质修复与立体视觉生成的场景，如老旧影片数字化。
立体视频生成近年来随着视频扩散模型的发展受到越来越多关注。然而，现有方法大多聚焦于从单目2D视频生成3D立体视频，通常假设输入视频质量较高，任务主要集中在填补变形视频中的遮挡区域，同时保持非遮挡区域。本文提出一种新流程，不仅生成立体视频，还能通过单一模型一致地增强左右视图视频。该方法通过对退化数据进行微调以实现修复，并利用扭曲掩码作为条件以保证立体生成的一致性。结果表明，该方法可在相对较小的合成立体视频数据集上进行微调，并应用于低质量的真实视频，同时完成立体视频生成与修复。实验显示，该方法在低分辨率输入下的立体视频生成上，无论定性还是定量均优于现有方法。
原文摘要 · Abstract (English)
Stereo video generation has been gaining increasing attention with recent advancements in video diffusion models. However, most existing methods focus on generating 3D stereoscopic videos from monocular 2D videos. These approaches typically assume that the input monocular video is of high quality, making the task primarily about inpainting occluded regions in the warped video while preserving disoccluded areas. In this paper, we introduce a new pipeline that not only generates stereo videos but also enhances both left-view and right-view videos consistently with a single model. Our approach achieves this by fine-tuning the model on degraded data for restoration, as well as conditioning the model on warped masks for consistent stereo generation. As a result, our method can be fine-tuned on a relatively small synthetic stereo video datasets and applied to low-quality real-world videos, performing both stereo video generation and restoration. Experiments demonstrate that our method outperforms existing approaches both qualitatively and quantitatively in stereo video generation from low-resolution inputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。