arXiv:2507.00429cs.CV2025-07ICCV被引 3

用分阶段扩散模型统一修复3D场景的几何与外观,支持灵活替换物体。

DiGA3D: Coarse-to-Fine Diffusional Propagation of Geometry and Appearance for Versatile 3D Inpainting

  • 多参考视图+注意力特征传播,保持跨视角外观一致。
  • 引入纹理-几何评分蒸馏损失,显著提升几何一致性。
  • 适用于物体移除、重绘、替换等多样化3D修复任务。

构建一个统一的文本引导3D修复框架,实现物体移除、重绘或替换的灵活操作至关重要。然而现有方法在统一框架中仍面临三大挑战:1)单参考视图方法在远离参考视图的视角上鲁棒性差;2)使用2D扩散先验独立修复多视图图像时出现外观不一致;3)当修复区域存在显著几何变化时,几何一致性受限。为此,我们提出DiGA3D,一种新颖且通用的3D修复框架,通过分阶段扩散机制传播一致的外观与几何信息。首先,设计稳健的多参考视图选择策略以降低传播误差;其次,提出注意力特征传播(AFP)机制,利用扩散模型将参考视图的注意力特征传递至其他视图,保障外观一致性;此外,引入纹理-几何评分蒸馏采样(TG-SDS)损失,进一步优化生成3D场景的几何一致性。在多个3D修复任务上的大量实验验证了方法的有效性。项目页面见 https://rorisis.github.io/DiGA3D/。

原文摘要 · Abstract (English)

Developing a unified pipeline that enables users to remove, re-texture, or replace objects in a versatile manner is crucial for text-guided 3D inpainting. However, there are still challenges in performing multiple 3D inpainting tasks within a unified framework: 1) Single reference inpainting methods lack robustness when dealing with views that are far from the reference view. 2) Appearance inconsistency arises when independently inpainting multi-view images with 2D diffusion priors; 3) Geometry inconsistency limits performance when there are significant geometric changes in the inpainting regions. To tackle these challenges, we introduce DiGA3D, a novel and versatile 3D inpainting pipeline that leverages diffusion models to propagate consistent appearance and geometry in a coarse-to-fine manner. First, DiGA3D develops a robust strategy for selecting multiple reference views to reduce errors during propagation. Next, DiGA3D designs an Attention Feature Propagation (AFP) mechanism that propagates attention features from the selected reference views to other views via diffusion models to maintain appearance consistency. Furthermore, DiGA3D introduces a Texture-Geometry Score Distillation Sampling (TG-SDS) loss to further improve the geometric consistency of inpainted 3D scenes. Extensive experiments on multiple 3D inpainting tasks demonstrate the effectiveness of our method. The project page is available at https://rorisis.github.io/DiGA3D/.

3D修复扩散模型几何一致性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。