用扩散模型统一实现高清虚化,解决手机拍摄低分辨率下的虚化难题。
Towards Photorealistic and Efficient Bokeh Rendering via Diffusion Framework

- 通过联合优化虚化与超分的扩散框架,一次完成图像增强和虚化
- 在低分辨率真实图像上生成逼真虚化效果,细节保留更完整
- 适合移动端实时虚化应用,尤其适用于高倍变焦场景
现有移动设备受限于紧凑光学设计(如小光圈),难以生成自然真实的虚化效果。尽管基于学习的方法已有进展,但在高倍数字变焦下仍面临分辨率下降、细节丢失的问题。传统先超分再虚化的两阶段方法效率低且易引入误差。为此,我们提出 MagicBokeh,一种统一的扩散框架,通过交替训练策略与聚焦感知掩码注意力机制,联合优化虚化渲染与超分辨率,显著提升控制力与视觉保真度。此外,引入退化感知深度模块,使低质量输入下的深度估计更准确。实验表明,MagicBokeh 能高效生成逼真的虚化效果,尤其在真实低分辨率图像上表现优异,为未来虚化渲染技术发展提供新路径。代码与模型已开源:https://github.com/vivoCameraResearch/MagicBokeh。
原文摘要 · Abstract (English)
Existing mobile devices are constrained by compact optical designs, such as small apertures, which make it difficult to produce natural, optically realistic bokeh effects. Although recent learning-based methods have shown promising results, they still struggle with photos captured under high digital zoom levels, which often suffer from reduced resolution and loss of fine details. A naive solution is to enhance image quality before applying bokeh rendering, yet this two-stage pipeline reduces efficiency and introduces unnecessary error accumulation. To overcome these limitations, we propose MagicBokeh, a unified diffusion-based framework designed for high-quality and efficient bokeh rendering. Through an alternative training strategy and a focus-aware masked attention mechanism, our method jointly optimizes bokeh rendering and super-resolution, substantially improving both controllability and visual fidelity. Furthermore, we introduce degradation-aware depth module to enable more accurate depth estimation from low-quality inputs. Experimental results demonstrate that MagicBokeh efficiently produces photorealistic bokeh effects, particularly on real-world low-resolution images, paving the way for future advancements in bokeh rendering. Our code and models are available at https://github.com/vivoCameraResearch/MagicBokeh.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。