arXiv:2509.26599cs.CV2025-09SIGGRAPH被引 5

让照片任意重新对焦,自由调节模糊程度和焦点位置。

DiffCamera: Arbitrary Refocusing on Images

  • 用扩散Transformer模型学习图像重对焦,模拟不同焦点下的成像效果。
  • 通过仿真生成大规模训练数据,解决真实配对数据难获取问题。
  • 引入叠加约束保证对焦结果符合物理规律,可正确合成多焦点图像。

景深(DoF)效应能带来美观的虚化,提升照片质量,但一旦成像后便无法修改,若主体失焦则难以补救。为此,我们提出DiffCamera,一个可基于任意新焦点和模糊程度灵活重对焦的模型。针对训练所需不同焦平面与散景水平的图像对难以获取的问题,我们构建了基于仿真的大规模数据生成管道。实验发现,仅使用基础扩散目标训练易导致错误的景深行为,因此我们提出一种堆叠约束:依据摄影原理,不同焦平面图像可线性融合为多焦点图像,该约束强制模型输出与场景结构及相机条件一致,确保重对焦结果可准确合成。我们还构建了基准测试评估模型性能。大量实验表明,DiffCamera在多种场景下均支持稳定重对焦,为摄影与生成式AI应用提供前所未有的景深调控能力。

原文摘要 · Abstract (English)

The depth-of-field (DoF) effect, which introduces aesthetically pleasing blur, enhances photographic quality but is fixed and difficult to modify once the image has been created. This becomes problematic when the applied blur is undesirable~(e.g., the subject is out of focus). To address this, we propose DiffCamera, a model that enables flexible refocusing of a created image conditioned on an arbitrary new focus point and a blur level. Specifically, we design a diffusion transformer framework for refocusing learning. However, the training requires pairs of data with different focus planes and bokeh levels in the same scene, which are hard to acquire. To overcome this limitation, we develop a simulation-based pipeline to generate large-scale image pairs with varying focus planes and bokeh levels. With the simulated data, we find that training with only a vanilla diffusion objective often leads to incorrect DoF behaviors due to the complexity of the task. This requires a stronger constraint during training. Inspired by the photographic principle that photos of different focus planes can be linearly blended into a multi-focus image, we propose a stacking constraint during training to enforce precise DoF manipulation. This constraint enhances model training by imposing physically grounded refocusing behavior that the focusing results should be faithfully aligned with the scene structure and the camera conditions so that they can be combined into the correct multi-focus image. We also construct a benchmark to evaluate the effectiveness of our refocusing model. Extensive experiments demonstrate that DiffCamera supports stable refocusing across a wide range of scenes, providing unprecedented control over DoF adjustments for photography and generative AI applications.

图像重对焦扩散模型景深控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。