用虚拟纹理对齐深度线索，实现高效单目3D重建
DreamTexture: Shape from Virtual Texture with Analysis by Augmentation
- 通过虚拟纹理对齐真实深度信息，利用扩散模型的单目几何理解
- 新共形映射优化使深度重建更轻量，避免高内存体素表示
- 提出'增广分析'新范式，适合追求效率的3D生成研究者
DreamFusion 开启了无监督3D重建的新范式，结合生成模型与可微渲染。然而其多视图渲染及大规模生成模型监督计算成本高且约束不足。本文提出 DreamTexture，一种基于虚拟纹理的单目3D重建方法：将虚拟纹理与输入图像中的真实深度线索对齐，利用现代扩散模型中编码的单目几何先验。随后通过新的共形映射优化从虚拟纹理变形中重建深度，缓解了内存密集型体素表示问题。实验表明，生成模型具备对单目形状线索的理解，可通过纹理增广与对齐提取——这一新范式称为‘增广分析’。
原文摘要 · Abstract (English)
DreamFusion established a new paradigm for unsupervised 3D reconstruction from virtual views by combining advances in generative models and differentiable rendering. However, the underlying multi-view rendering, along with supervision from large-scale generative models, is computationally expensive and under-constrained. We propose DreamTexture, a novel Shape-from-Virtual-Texture approach that leverages monocular depth cues to reconstruct 3D objects. Our method textures an input image by aligning a virtual texture with the real depth cues in the input, exploiting the inherent understanding of monocular geometry encoded in modern diffusion models. We then reconstruct depth from the virtual texture deformation with a new conformal map optimization, which alleviates memory-intensive volumetric representations. Our experiments reveal that generative models possess an understanding of monocular shape cues, which can be extracted by augmenting and aligning texture cues -- a novel monocular reconstruction paradigm that we call Analysis by Augmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。