arXiv:2504.05400cs.CVcs.AI2025-04ICCV被引 11

用真实碎片数据训练3D拼合模型,提升对未知物体的泛化能力。

GARF: Learning Generalizable 3D Reassembly for Real-World Fractures

  • 通过断裂感知预训练学习碎片特征,结合流匹配实现精确6自由度对齐
  • 在真实数据上达到82.87%旋转误差降低和25.15%零件准确率提升
  • 适合考古、古人类学等需要真实破碎物复原的研究者使用

3D重装是一项具有广泛科学应用价值的空间智能挑战。尽管大规模合成数据集推动了基于学习的方法发展,但其在不同领域间的泛化能力仍受限。关键问题是:在合成数据上训练的模型能否泛化到真实断裂场景——后者断口模式更复杂。为此,我们提出GARF,一种面向真实断裂的可泛化3D重装框架。GARF利用断裂感知预训练从单个碎片中学习断裂特征,通过流匹配实现精确6-DoF对齐。推理时引入一步预装配,增强对未见物体和不同碎片数量的鲁棒性。与考古学家、古人类学家及鸟类学家合作,我们构建了Fractura数据集,涵盖陶瓷、骨骼、蛋壳和石器等多种真实断裂类型。全面实验表明,该方法在合成与真实数据集上均显著优于现有最优方法,实现82.87%更低的旋转误差与25.15%更高的零件准确率。这表明基于合成数据训练可有效推动真实世界3D拼图求解,并在未见物体形状与多样断裂类型间展现强泛化能力。GARF代码、数据与演示已开源。

原文摘要 · Abstract (English)

3D reassembly is a challenging spatial intelligence task with broad applications across scientific domains. While large-scale synthetic datasets have fueled promising learning-based approaches, their generalizability to different domains is limited. Critically, it remains uncertain whether models trained on synthetic datasets can generalize to real-world fractures where breakage patterns are more complex. To bridge this gap, we propose GARF, a generalizable 3D reassembly framework for real-world fractures. GARF leverages fracture-aware pretraining to learn fracture features from individual fragments, with flow matching enabling precise 6-DoF alignments. At inference time, we introduce one-step preassembly, improving robustness to unseen objects and varying numbers of fractures. In collaboration with archaeologists, paleoanthropologists, and ornithologists, we curate Fractura, a diverse dataset for vision and learning communities, featuring real-world fracture types across ceramics, bones, eggshells, and lithics. Comprehensive experiments have shown our approach consistently outperforms state-of-the-art methods on both synthetic and real-world datasets, achieving 82.87\% lower rotation error and 25.15\% higher part accuracy. This sheds light on training on synthetic data to advance real-world 3D puzzle solving, demonstrating its strong generalization across unseen object shapes and diverse fracture types. GARF's code, data and demo are available at https://ai4ce.github.io/GARF/.

3D重装断裂分析泛化能力真实数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。