用模糊图像还原拍摄瞬间及前后动态场景
Generating the Past, Present and Future from a Motion-Blurred Image
- 利用预训练视频扩散模型从模糊图中重建时空动态
- 可恢复拍摄时、前后瞬间的复杂运动过程，精度优于现有方法
- 适合需要恢复真实世界动态的视觉重建与运动分析任务
我们探讨一个核心问题：一张运动模糊图像能揭示场景的过去、现在和未来吗？尽管运动模糊会损失细节并降低视觉质量，但它也编码了曝光期间场景与相机的运动信息。以往方法依赖手工先验或特定网络结构来解决这一逆问题中的歧义，且未引入大规模数据集上的图像与视频先验，导致难以还原复杂场景动态，也无法推断图像拍摄前后的事件。本文提出一种新方法，重新利用在互联网规模数据上预训练的视频扩散模型，从模糊图像中重建出拍摄瞬间及前后可能发生的动态视频序列。该方法鲁棒性强、泛化能力好，可处理真实复杂场景，支持相机轨迹恢复、物体运动推断和动态三维场景重建等下游任务。代码与数据已公开于 https://blur2vid.github.io
原文摘要 · Abstract (English)
We seek to answer the question: what can a motion-blurred image reveal about a scene's past, present, and future? Although motion blur obscures image details and degrades visual quality, it also encodes information about scene and camera motion during an exposure. Previous techniques leverage this information to estimate a sharp image from an input blurry one, or to predict a sequence of video frames showing what might have occurred at the moment of image capture. However, they rely on handcrafted priors or network architectures to resolve ambiguities in this inverse problem, and do not incorporate image and video priors on large-scale datasets. As such, existing methods struggle to reproduce complex scene dynamics and do not attempt to recover what occurred before or after an image was taken. Here, we introduce a new technique that repurposes a pre-trained video diffusion model trained on internet-scale datasets to recover videos revealing complex scene dynamics during the moment of capture and what might have occurred immediately into the past or future. Our approach is robust and versatile; it outperforms previous methods for this task, generalizes to challenging in-the-wild images, and supports downstream tasks such as recovering camera trajectories, object motion, and dynamic 3D scene structure. Code and data are available at https://blur2vid.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。