用扩散模型+Transformer提升3D重建细节与多视角一致性
DT-NeRF: A Diffusion and Transformer-Based Optimization Approach for Neural Radiance Fields in 3D Reconstruction
- 融合扩散模型与Transformer优化神经辐射场
- 在Matterport3D和ShapeNet上显著提升PSNR、SSIM等指标
- 适合需要高精度3D重建的研究者与工业应用
本文提出一种基于扩散模型与Transformer的神经辐射场优化方法（DT-NeRF），旨在提升3D场景重建中的细节恢复能力与多视角一致性。通过结合扩散模型与Transformer，DT-NeRF在稀疏视角下仍能有效还原细节，并在复杂几何场景中保持高精度。实验表明，DT-NeRF在Matterport3D和ShapeNet数据集上的表现显著优于传统NeRF及其他先进方法，尤其在PSNR、SSIM、Chamfer Distance和Fidelity等指标上优势明显。消融实验进一步验证了扩散模块与Transformer模块对性能的关键作用，二者缺失均导致性能下降。该设计展示了模块间的协同效应，为3D场景重建提供了高效准确的解决方案。未来可探索更先进的生成模型与网络架构，以提升其在大规模动态场景中的表现。
原文摘要 · Abstract (English)
This paper proposes a Diffusion Model-Optimized Neural Radiance Field (DT-NeRF) method, aimed at enhancing detail recovery and multi-view consistency in 3D scene reconstruction. By combining diffusion models with Transformers, DT-NeRF effectively restores details under sparse viewpoints and maintains high accuracy in complex geometric scenes. Experimental results demonstrate that DT-NeRF significantly outperforms traditional NeRF and other state-of-the-art methods on the Matterport3D and ShapeNet datasets, particularly in metrics such as PSNR, SSIM, Chamfer Distance, and Fidelity. Ablation experiments further confirm the critical role of the diffusion and Transformer modules in the model's performance, with the removal of either module leading to a decline in performance. The design of DT-NeRF showcases the synergistic effect between modules, providing an efficient and accurate solution for 3D scene reconstruction. Future research may focus on further optimizing the model, exploring more advanced generative models and network architectures to enhance its performance in large-scale dynamic scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。