arXiv:2511.21422cs.CV2025-11被引 2

融合几何与颜色信息,提升碎片化3D物体重装精度

E-M3RF: An Equivariant Multimodal 3D Re-assembly Framework

  • 用旋转等变编码器和Transformer融合点位与颜色特征
  • 在真实文物数据集上旋转误差降23.1%,平移误差降13.2%
  • 适合处理小、对称或破损碎片的重装任务

3D重装是基础几何问题,近年逐渐由深度学习方法取代传统优化。现有方法多依赖几何特征,面对小块、侵蚀或对称碎片时易失效,且未显式约束物理不重叠。为此,提出E-M3RF框架,输入含位置与颜色的点云,通过SE(3)流匹配预测重装变换。每个碎片同时编码几何与颜色:① 使用旋转等变编码器提取一致几何特征;② 用Transformer编码点处颜色。两组特征融合形成多模态表示。在四个数据集(Breaking Bad、Fantastic Breaks、RePAIR、Presious)上测试,E-M3RF在RePAIR数据集上旋转误差降低23.1%,平移误差降低13.2%,Chamfer Distance减少18.4%。

原文摘要 · Abstract (English)

3D reassembly is a fundamental geometric problem, and in recent years it has increasingly been challenged by deep learning methods rather than classical optimization. While learning approaches have shown promising results, most still rely primarily on geometric features to assemble a whole from its parts. As a result, methods struggle when geometry alone is insufficient or ambiguous, for example, for small, eroded, or symmetric fragments. Additionally, solutions do not impose physical constraints that explicitly prevent overlapping assemblies. To address these limitations, we introduce E-M3RF, an equivariant multimodal 3D reassembly framework that takes as input the point clouds, containing both point positions and colors of fractured fragments, and predicts the transformations required to reassemble them using SE(3) flow matching. Each fragment is represented by both geometric and color features: i) 3D point positions are encoded as rotationconsistent geometric features using a rotation-equivariant encoder, ii) the colors at each 3D point are encoded with a transformer. The two feature sets are then combined to form a multimodal representation. We experimented on four datasets: two synthetic datasets, Breaking Bad and Fantastic Breaks, and two real-world cultural heritage datasets, RePAIR and Presious, demonstrating that E-M3RF on the RePAIR dataset reduces rotation error by 23.1% and translation error by 13.2%, while Chamfer Distance decreases by 18.4% compared to competing methods.

3D重装多模态等变网络文物修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。