自监督循环框架提升无人机视角合成与3D重建精度
A self-supervised cyclic neural-analytic approach for novel view synthesis and 3D reconstruction
- 融合神经渲染与解析几何的循环架构，无需标注数据
- 在稀疏区域和未见场景中显著改善图像与网格重建质量
- 适合复杂户外环境下的自主飞行系统开发
从视频生成新视角对实现自主无人机导航至关重要。尽管神经渲染技术推动了新轨迹生成方法的发展，但这些方法在缺乏优化飞行路径时，难以泛化到远离训练数据的区域，导致重建效果不佳。本文提出一种自监督循环神经-解析框架，结合高质量神经渲染输出与解析方法提供的精确几何信息。该方案显著提升了新视角合成中的RGB图像和网格重建质量，尤其在数据稀疏区域及与训练集差异较大的区域表现更优。采用基于Transformer的架构进行图像重建，可有效适应未见姿态，无需依赖大规模标注数据。实验表明，本方法在新视角渲染与3D重建方面均取得显著提升，据我们所知为首次实现，为复杂户外环境中自主导航树立了新标准。
原文摘要 · Abstract (English)
Generating novel views from recorded videos is crucial for enabling autonomous UAV navigation. Recent advancements in neural rendering have facilitated the rapid development of methods capable of rendering new trajectories. However, these methods often fail to generalize well to regions far from the training data without an optimized flight path, leading to suboptimal reconstructions. We propose a self-supervised cyclic neural-analytic pipeline that combines high-quality neural rendering outputs with precise geometric insights from analytical methods. Our solution improves RGB and mesh reconstructions for novel view synthesis, especially in undersampled areas and regions that are completely different from the training dataset. We use an effective transformer-based architecture for image reconstruction to refine and adapt the synthesis process, enabling effective handling of novel, unseen poses without relying on extensive labeled datasets. Our findings demonstrate substantial improvements in rendering views of novel and also 3D reconstruction, which to the best of our knowledge is a first, setting a new standard for autonomous navigation in complex outdoor environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。