arXiv:2510.03110cs.CV2025-10NeurIPS被引 9

用3D结构信息指导图像补全,让生成内容更符合真实几何关系。

GeoComplete: Geometry-Aware Diffusion for Reference-Driven Image Completion

  • 通过投影点云条件化扩散过程,注入几何信息
  • 目标感知掩码使模型聚焦有效参考线索,提升复杂场景表现
  • 在几何一致性上显著优于现有方法,适合需要精确结构重建的任务

参考驱动的图像补全旨在利用额外图像恢复目标视图中的缺失区域,当目标视图与参考图像差异较大时尤为困难。现有生成方法仅依赖扩散先验,缺乏相机位姿或深度等几何线索,常导致内容错位或不真实。我们提出GeoComplete,一种引入显式三维结构引导的新型框架,以确保补全区域的几何一致性。该框架包含两个核心设计:将扩散过程条件化于投影点云以融入几何信息,以及采用目标感知掩码引导模型关注相关参考线索。其双分支扩散架构中,一支从掩码目标生成缺失内容,另一支从投影点云提取几何特征,跨分支联合自注意力机制保证生成一致准确。针对参考中可见但目标中缺失的区域,我们将在每个参考中投影目标视图以检测遮挡区域,并在训练中进行掩码处理。该策略引导模型聚焦有效信息,显著提升复杂场景下的性能。实验表明,GeoComplete相较最先进方法提升17.1 PSNR,大幅增强几何准确性,同时保持高视觉质量。

原文摘要 · Abstract (English)

Reference-driven image completion, which restores missing regions in a target view using additional images, is particularly challenging when the target view differs significantly from the references. Existing generative methods rely solely on diffusion priors and, without geometric cues such as camera pose or depth, often produce misaligned or implausible content. We propose GeoComplete, a novel framework that incorporates explicit 3D structural guidance to enforce geometric consistency in the completed regions, setting it apart from prior image-only approaches. GeoComplete introduces two key ideas: conditioning the diffusion process on projected point clouds to infuse geometric information, and applying target-aware masking to guide the model toward relevant reference cues. The framework features a dual-branch diffusion architecture. One branch synthesizes the missing regions from the masked target, while the other extracts geometric features from the projected point cloud. Joint self-attention across branches ensures coherent and accurate completion. To address regions visible in references but absent in the target, we project the target view into each reference to detect occluded areas, which are then masked during training. This target-aware masking directs the model to focus on useful cues, enhancing performance in difficult scenarios. By integrating a geometry-aware dual-branch diffusion architecture with a target-aware masking strategy, GeoComplete offers a unified and robust solution for geometry-conditioned image completion. Experiments show that GeoComplete achieves a 17.1 PSNR improvement over state-of-the-art methods, significantly boosting geometric accuracy while maintaining high visual quality.

图像补全扩散模型3D几何

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。