arXiv:2503.24374cs.CV2025-03CVPR

用少至5张无姿态照片,高效还原真实场景新视角。

ERUPT: Efficient Rendering with Unposed Patch Transformer

  • 用图像块查询替代像素点,降低渲染计算量。
  • 在商用硬件上实现600帧每秒,比当前方法快10倍。
  • 适合真实街景等缺乏精确相机位姿的场景重建。

本文针对从少量RGB图像中合成新视角的问题,提出ERUPT(Efficient Rendering with Unposed Patch Transformer)模型,可在无姿态图像下实现高效场景重建。通过引入基于图像块的查询机制,相比传统像素级查询显著减少计算开销,使模型在训练和推理阶段均具备高效率,可在商用硬件上实现600 fps的渲染速度。该模型采用学习得到的隐式相机位姿,支持使用稀疏或不准确标注位姿的数据集进行训练。实验表明,其能在大规模真实世界数据上良好泛化,并构建了新的基准数据集MSVS-1M,用于基于街景影像的隐式视图合成。与NeRF和Gaussian Splatting相比,ERUPT仅需5张无姿态输入图像即可生成高质量新视角,相较于现有方法减少约95%标注数据需求,计算成本降低一个数量级,实现了对多样化真实场景的高效新视角合成。

原文摘要 · Abstract (English)

This work addresses the problem of novel view synthesis in diverse scenes from small collections of RGB images. We propose ERUPT (Efficient Rendering with Unposed Patch Transformer) a state-of-the-art scene reconstruction model capable of efficient scene rendering using unposed imagery. We introduce patch-based querying, in contrast to existing pixel-based queries, to reduce the compute required to render a target view. This makes our model highly efficient both during training and at inference, capable of rendering at 600 fps on commercial hardware. Notably, our model is designed to use a learned latent camera pose which allows for training using unposed targets in datasets with sparse or inaccurate ground truth camera pose. We show that our approach can generalize on large real-world data and introduce a new benchmark dataset (MSVS-1M) for latent view synthesis using street-view imagery collected from Mapillary. In contrast to NeRF and Gaussian Splatting, which require dense imagery and precise metadata, ERUPT can render novel views of arbitrary scenes with as few as five unposed input images. ERUPT achieves better rendered image quality than current state-of-the-art methods for unposed image synthesis tasks, reduces labeled data requirements by ~95\% and decreases computational requirements by an order of magnitude, providing efficient novel view synthesis for diverse real-world scenes.

新视角合成无姿态图像高效渲染街景重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。