让单图生成的3D模型多视角更一致,效果接近顶尖水平
View-Adaptive Renderer for View-Consistent 2D-to-3D Generation
- 用视点自适应渲染器分别修正不同角度的误差
- 在无扩散模型监督下实现接近顶尖的3D重建精度
- 适合需要高效高质3D生成的工业应用
从单张图像重建3D形状是计算机视觉中的基础挑战。传统方法先从单图生成多视角再用NeRF重建,但投影模糊常导致视角间视觉不连续,影响3D模型准确性。现有方案或计算开销大,或无法有效解决视角不一致问题。为此,我们提出一种视点自适应神经渲染框架,可在部分不一致的多视角输入下仍实现鲁棒3D重建。方法引入视点自适应渲染器,独立修正视角相关误差,同时共享全局特征主干以保持结构一致性。此外,设计自注意力融合模块,自适应整合多视角信息,确保几何一致性,无需依赖复杂的间接正则化或高耗计算方法。大量实验表明,该方法持续提升3D重建保真度。重要的是,无需扩散模型的SDS监督,仅用光度渲染损失与轻量级注意力正则器,即达到近顶尖性能。该方法在准确率与效率间取得良好平衡,适用于真实场景应用。
原文摘要 · Abstract (English)
Reconstructing 3D shapes from a single image remains a fundamental yet challenging problem in computer vision. Traditional monocular 3D generation pipelines typically synthesize multiple views from a single input image before applying Neural Radiance Field (NeRF)-based reconstruction. However, inherent projective ambiguities often produce visual discontinuities across generated viewpoints, leading to inaccuracies in reconstructed 3D models. Current solutions either incur significant additional computational burdens or fail to adequately resolve practical inconsistencies between synthesized views. To address these limitations, we propose a novel viewpoint-adaptive neural rendering framework that enables robust 3D reconstruction even when given partially inconsistent multi-view inputs. Our approach introduces view-adaptive neural renderers that independently correct viewpoint-dependent errors while simultaneously sharing a global feature backbone to preserve structural coherence. Furthermore, we propose a self-attention fusion module that adaptively integrates multi-view information, ensuring geometric consistency without relying heavily on indirect regularizations or computationally intensive methods. Through extensive experiments, we demonstrate that our method consistently improves 3D reconstruction fidelity. Importantly, our approach achieves near state-of-the-art performance without diffusion-based SDS supervision, relying primarily on photometric rendering loss with lightweight attention regularizers. This balance between accuracy and efficiency makes the proposed framework highly practical for real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。