用光流引导扩散模型,让细微运动放大更真实清晰。
GeoDiffMM: Geometry-Guided Conditional Diffusion for Motion Magnification
- 以光流为几何先验,指导扩散过程放大运动。
- 在真实与合成数据上均超越现有方法,显著提升放大效果。
- 适合需要高保真运动放大的医学、工业检测场景。
视频运动放大(VMM)可将微小的宏观运动增强至可感知水平。现有主流欧拉方法通过解耦纹理、形状和频率等表征来缓解放大引入的噪声,但在运动位移极小时仍难以消除光子噪声对真实微运动的干扰。本文提出GeoDiffMM,一种基于扩散模型的拉格朗日式VMM框架,以光流作为几何线索,实现结构一致的运动放大。我们设计了无噪声光流增强策略,合成多样非刚性运动场以无光子噪声为监督,帮助模型学习更准确的几何感知光流并提升泛化能力。随后,提出扩散运动放大器,其去噪过程同时依赖(i)光流作为几何先验,(ii)可学习的放大因子控制幅度,从而选择性放大与场景语义和结构一致的运动成分。最后,通过基于光流的视频生成,将放大后的运动以高保真度映射回图像域。在真实与合成数据集上的大量实验表明,GeoDiffMM优于当前最优方法,显著提升运动放大性能。
原文摘要 · Abstract (English)
Video Motion Magnification (VMM) amplifies subtle macroscopic motions to a perceptible level. Recently, existing mainstream Eulerian approaches address amplification-induced noise via decoupling representation learning such as texture, shape and frequency schemes, but they still struggle to mitigate the interference of photon noise on true micro-motion when motion displacements are very small. We propose GeoDiffMM, a novel diffusion-based Lagrangian VMM framework conditioned on optical flow as a geometric cue, enabling structurally consistent motion magnification. Specifically, we design a Noise-Free Optical Flow Augmentation strategy that synthesizes diverse nonrigid motion fields without photon noise as supervision, helping the model learn more accurate geometry-aware optical flow and generalize better. Next, we develop a Diffusion Motion Magnifier that conditions the denoising process on (i) optical flow as a geometry prior and (ii) a learnable magnification factor controlling magnitude, thereby selectively amplifying motion components consistent with scene semantics and structure. Finally, we perform Flow-based Video Synthesis to map the amplified motion back to the image domain with high fidelity. Extensive experiments on real and synthetic datasets show that GeoDiffMM outperforms state-of-the-art methods and significantly improves motion magnification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。