仅用单张图像训练，实现野外场景的高质量新视角生成
One View Is Enough! Monocular Training for In-the-Wild Novel View Generation
- 用单图生成伪目标视图，通过深度估计搭建几何框架
- 在3000万张无标注图片上训练，零样本下超越现有方法
- 推理无需深度或3D结构，速度比最优基线快600倍
单目新视角合成传统依赖多视角图像对进行监督，限制了训练数据规模与多样性。我们提出仅需单张图像即可训练：OVIE模型完全基于未配对的互联网图像进行训练。训练时利用单目深度估计器作为几何支撑，将源图像升维至3D，施加采样相机变换后投影生成伪目标视图。为处理遮挡问题，引入掩码训练机制，仅在有效区域计算几何、感知和纹理损失，支持在3000万张未经筛选的图像上训练。推理阶段完全无需深度估计或3D表示。仅在野外图像上训练的OVIE，在零样本设置下表现优于以往方法，且速度比第二优基线快600倍。代码与模型已公开于https://github.com/AdrienRR/ovie。
原文摘要 · Abstract (English)
Monocular novel-view synthesis has long required multi-view image pairs for supervision, limiting training data scale and diversity. We argue it is not necessary: one view is enough. We present OVIE, trained entirely on unpaired internet images. We leverage a monocular depth estimator as a geometric scaffold at training time: we lift a source image into 3D, apply a sampled camera transformation, and project to obtain a pseudo-target view. To handle disocclusions, we introduce a masked training formulation that restricts geometric, perceptual, and textural losses to valid regions, enabling training on 30 million uncurated images. At inference, OVIE is geometry-free, requiring no depth estimator or 3D representation. Trained exclusively on in-the-wild images, OVIE outperforms prior methods in a zero-shot setting, while being 600x faster than the second-best baseline. Code and models are publicly available at https://github.com/AdrienRR/ovie.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。