无需对齐即可联合重建多个物体,利用重复模式提升3D结构还原精度。
JRM: Joint Reconstruction Model for Multiple Objects without Alignment
- 通过隐式潜空间聚合未对齐观测,实现个性化生成式重建。
- 在合成与真实数据上均优于独立与对齐基线模型,提升鲁棒性。
- 适合处理非刚性形变,如关节运动,无需人工匹配与对齐。
以物体为中心的重建旨在通过独立物体的组合恢复场景的3D结构。尽管这种独立性简化了建模,却忽略了有助于提升重建质量的关键信号,如同一物体在场景中多次出现或跨扫描重复的情况。本文提出联合重建模型(JRM),通过将物体重建视为个性化生成问题:多个观测共享一个共同主体,需保持一致性,同时遵循每个观测的具体姿态与状态。现有方法依赖显式匹配与刚性对齐,对误差敏感且难以扩展至非刚性变换。相比之下,JRM是一种3D流匹配生成模型,能隐式聚合未对齐观测,在潜空间中学习生成一致且忠实的重建结果,无需显式约束。在合成与真实数据上的评估表明,JRM的隐式聚合消除了显式对齐需求,提升了对错误关联的鲁棒性,并自然处理如关节运动等非刚性变化。总体而言,JRM在重建质量上优于独立与对齐基线模型。
原文摘要 · Abstract (English)
Object-centric reconstruction seeks to recover the 3D structure of a scene through composition of independent objects. While this independence can simplify modeling, it discards strong signals that could improve reconstruction, notably repetition where the same object model is seen multiple times in a scene, or across scans. We propose the Joint Reconstruction Model (JRM) to leverage repetition by framing object reconstruction as one of personalized generation: multiple observations share a common subject that should be consistent for all observations, while still adhering to the specific pose and state from each. Prior methods in this direction rely on explicit matching and rigid alignment across observations, making them sensitive to errors and difficult to extend to non-rigid transformations. In contrast, JRM is a 3D flow-matching generative model that implicitly aggregates unaligned observations in its latent space, learning to produce consistent and faithful reconstructions in a data-driven manner without explicit constraints. Evaluations on synthetic and real-world data show that JRM's implicit aggregation removes the need for explicit alignment, improves robustness to incorrect associations, and naturally handles non-rigid changes such as articulation. Overall, JRM outperforms both independent and alignment-based baselines in reconstruction quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。