用几何一致性提升多视角材质估计的稳定性
Geo-ID: Test-Time Geometric Consensus for Cross-View Consistent Intrinsics
- 通过稀疏几何对应构建不确定性感知的共识目标,统一多视角预测
- 视图越多一致性越强,单视角效果与原有模型相当
- 无需重训练,适配现有材质分解模型,适合3D重建与编辑
内在图像分解旨在从图像中估计物理渲染参数,如反照率、粗糙度和金属度。尽管近期方法在单视角下表现优异,但对同一场景的多视角独立应用常导致估计不一致,限制其在可编辑神经场景和三维重建等下游任务中的使用。视频类模型虽能提升跨帧一致性,但需密集有序序列和大量计算,难以适用于稀疏无序图像集。我们提出Geo-ID,一种新型测试时框架,通过稀疏几何对应将独立的单视角预测耦合为不确定性感知的共识目标,使预训练的单视角预测器实现跨视角一致性。该方法模型无关,无需微调或逆渲染,可直接应用于现成的内在图像预测器。在合成基准和真实场景上的实验表明,随着视角数增加,跨视角一致性显著提升,同时保持与原模型相当的单视角分解性能。进一步验证了生成的一致内在参数可支持下游神经场景表示中的连贯外观编辑与再光照。
原文摘要 · Abstract (English)
Intrinsic image decomposition aims to estimate physically based rendering (PBR) parameters such as albedo, roughness, and metallicity from images. While recent methods achieve strong single-view predictions, applying them independently to multiple views of the same scene often yields inconsistent estimates, limiting their use in downstream applications such as editable neural scenes and 3D reconstruction. Video-based models can improve cross-frame consistency but require dense, ordered sequences and substantial compute, limiting their applicability to sparse, unordered image collections. We propose Geo-ID, a novel test-time framework that repurposes pretrained single-view intrinsic predictors to produce cross-view consistent decompositions by coupling independent per-view predictions through sparse geometric correspondences that form uncertainty-aware consensus targets. Geo-ID is model-agnostic, requires no retraining or inverse rendering, and applies directly to off-the-shelf intrinsic predictors. Experiments on synthetic benchmarks and real-world scenes demonstrate substantial improvements in cross-view intrinsic consistency as the number of views increases, while maintaining comparable single-view decomposition performance. We further show that the resulting consistent intrinsics enable coherent appearance editing and relighting in downstream neural scene representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。