用额外视角提升单图3D重建质量,无需重训练即可增强精度与一致性。
When Does An Extra View Help? Adapting Single-View 3D Reconstruction with Extra Imagery

- 引入零样本与优化适配策略,利用附加图像改进单图重建。
- 在多个数据集上显著提升重建准确率和跨视图一致性。
- 适合希望在不重训模型前提下增强3D重建性能的研究者。
从单张图像重建3D物体是计算机视觉中的挑战性问题,主要因视角信息缺失导致结构不完整。引入额外视图可缓解此问题,但现有方法缺乏将附加视图有效融入单图重建机制的方案。为此,我们提出ASV3D框架,可在测试时利用一张额外图像适配单视图3D重建。设计两种适配策略:(i) 零样本适配,无需重训练即可提升重建质量;(ii) 优化适配,通过对比学习进一步增强视觉保真度与跨视图一致性。我们在两个主流单视图3D重建流水线中应用ASV3D,涵盖基准与真实世界数据集。结果表明,该方法在无约束多视图输入下持续提升重建准确率与鲁棒性,在定量指标与人工偏好评价中均优于基线。代码与真实世界数据集已开源。
原文摘要 · Abstract (English)
Reconstruction of 3D objects from a single image is a challenging research problem in computer vision. The key challenge is the lack of critical information from viewpoints to complete 3D structures. Using an additional view may help to resolve the issue. However, there is no mechanism that can integrate the extra view into the single-view 3D reconstruction principle. We address this challenge by proposing ASV3D, a framework for adapting single-view 3D object reconstruction to test-time data with support from one additional image. We introduce two adaptation strategies: (i) a zero-shot adaptation scheme that leverages the auxiliary image to improve the reconstruction quality of an object without retraining, and (ii) an optimised adaptation scheme that further enhances visual fidelity and cross-view consistency via contrastive learning. We apply our ASV3D to improve two state-of-the-art single-view 3D reconstruction pipelines on both benchmark and real-world datasets. Results demonstrate that our approach consistently improves reconstruction accuracy and robustness under unconstrained multi-view inputs, outperforming the baselines in both quantitative metrics and human preference. We publish our code and the real-world object dataset in our project page at https://github.com/YNhuHuynh/ASV3D/tree/main.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。