无需训练,用几何变换实现生成3D模型与单目观测的精准对齐。
Robust 3D Alignment of Generative Reconstructions via Partial Monocular Observations

- 通过显式尺度因子和粗到精策略,解决输入稀疏与生成模型间的尺度模糊问题。
- 在极端条件下仍能稳定对齐,比传统方法和主流学习模型提升显著。
- 适合需要高精度3D对齐的视觉重建、AR/VR应用,尤其适用于低质量输入场景。
将生成式3D重建结果与部分单目观测进行对齐,是计算机视觉中一个关键但研究不足的挑战。该任务因单目输入噪声大、稀疏,而生成先验密集且存在尺度模糊与几何幻觉,加之初始重叠缺失,导致传统注册流程失效。为此,我们提出一种无需训练、可解释的几何对齐框架,通过3D相似变换(Sim(3))将生成先验锚定,恢复精确的度量尺度与姿态。具体地,引入显式尺度因子消除尺度歧义,采用粗到精对齐策略,利用几何感知描述子实现鲁棒初始化,并设计解耦闭式求解器进行精修。此外,引入幻觉过滤机制有效抑制由幻觉几何产生的异常点。为评估极端条件下的对齐性能,我们构建了GenPMOAlign--Where2Place基准,专门用于生成式到部分单目观测的对齐测试。实验表明,本方法在稳定性与准确性上均显著优于经典几何管线与最先进学习基线。代码与基准将公开发布。
原文摘要 · Abstract (English)
Aligning generative 3D reconstructions with partial monocular observations is a critical but under-explored challenge in computer vision. This task is inherently ill-posed due to severe asymmetries between noisy, sparse monocular inputs and dense generative priors, whose scale ambiguity and geometric hallucinations, combined with the lack of initial overlap, render traditional registration pipelines ineffective. To resolve these issues, we propose a training-free and interpretable geometric alignment framework that grounds generative 3D priors via a 3D similarity transformation (Sim(3)), which can recover accurate metric scale and pose. Specifically, we introduce an explicit scale factor to resolve metric ambiguity and employ a coarse-to-fine alignment strategy, leveraging geometry-aware descriptors for robust initialization and a decoupled closed-form solver for precision refinement. In addition, we introduce a Hallucination Filtering operation to effectively suppress outliers caused by hallucinated geometry. To evaluate alignment performance under these extreme conditions, we introduce GenPMOAlign--Where2Place, a rigorous benchmark specifically designed for Generative-to-Partial Monocular Observational Alignment. Experiments demonstrate that our method achieves stable and accurate registration, substantially outperforming both classical geometric pipelines and state-of-the-art learning-based baselines. Code and the benchmark will be publicly released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。