arXiv:2503.18254cs.CVcs.GR2025-03SIGGRAPH被引 7

让3D形状的语义特征区分左右手等相似部位,提升匹配精度。

Surface-Aware Distilled 3D Semantic Features

  • 构建表面对齐的嵌入空间,用对比损失分离表面远端特征。
  • 在无配对数据下仅需少量网格即可生成鲁棒特征,支持低数据场景。
  • 适用于纹理转移、姿态对齐等任务,无需额外优化即可泛化到新形状。

许多3D任务如姿态对齐、动画迁移和3D重建依赖于3D形状间的对应关系建立。近期方法通过预训练视觉模型提取的语义特征进行成对匹配,但这些特征难以区分同一语义类别中的实例(如左/右手),导致显著映射误差。为此,我们学习一种表面对齐的嵌入空间,使其对这类歧义具有鲁棒性,并支持整个3D形状家族的共享映射。该方法为自监督学习,仅需少量未配对训练网格,即可在测试时为新且可能不完整的3D形状推断特征。我们引入对比损失,保留来自基础模型的语义内容,同时区分位于形状表面较远位置的特征。在对应匹配基准上表现更优,并成功应用于2D-to-3D与3D-to-3D纹理迁移、部分分割、姿态对齐及低数据条件下的运动迁移。与以往成对方法不同,我们的方案构建联合嵌入空间,使已见与未见3D形状隐式对齐,无需进一步优化。代码已公开于https://graphics.tudelft.nl/SurfaceAware3DFeatures。

原文摘要 · Abstract (English)

Many 3D tasks such as pose alignment, animation, motion transfer, and 3D reconstruction rely on establishing correspondences between 3D shapes. This challenge has recently been approached by pairwise matching of semantic features from pre-trained vision models. However, despite their power, these features struggle to differentiate instances of the same semantic class such as ``left hand'' versus ``right hand'' which leads to substantial mapping errors. To solve this, we learn a surface-aware embedding space that is robust to these ambiguities while facilitating shared mapping for an entire family of 3D shapes. Importantly, our approach is self-supervised and requires only a small number of unpaired training meshes to infer features for new possibly imperfect 3D shapes at test time. We achieve this by introducing a contrastive loss that preserves the semantic content of the features distilled from foundational models while disambiguating features located far apart on the shape's surface. We observe superior performance in correspondence matching benchmarks and enable downstream applications including 2D-to-3D and 3D-to-3D texture transfer, in-part segmentation, pose alignment, and motion transfer in low-data regimes. Unlike previous pairwise approaches, our solution constructs a joint embedding space, where both seen and unseen 3D shapes are implicitly aligned without further optimization. The code is available at https://graphics.tudelft.nl/SurfaceAware3DFeatures.

3D特征表面对齐自监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。