用谱域神经映射实现真实图像到3D模型的高保真属性迁移
RealSkin: Spatio-Spectral Partial Neural Adjoint Maps for Image-to-3D Attribute Transfer
- 在学习的谱空间中优化对应关系，融合空间与谱信息
- 支持部分对应和非等距变形，在拓扑差异大时仍有效
- 适合影视游戏中的真实感3D资产生成，无需人工标注
构建逼真的3D资产需弥合真实观测与合成模型之间的外观差异。一种有前景的方法是将真实图像的视觉属性迁移到合成3D表面。传统方法在分辨率不匹配和点对应离散性方面表现不佳。相比之下，基于函数映射的方法虽能实现平滑属性传播，但依赖近等距假设与拓扑一致性。为克服这些限制，我们提出RealSkin——一种自监督框架，通过空间对应引导，在学习的谱域中进行对应优化。首先引入空间引导注册算法，在严重拓扑差异下建立粗略对应。为进一步放松严格等距假设并处理部分对应，设计了谱感知神经伴随网络，将部分对应嵌入神经函数空间，并建模非等距残差以优化对应关系。实验表明，该方法在挑战性的真实到合成场景中达到当前最优性能。代码将公开发布。
原文摘要 · Abstract (English)
Creating photorealistic 3D assets requires bridging the appearance gap between real-world observations and synthetic models. A promising approach is to transfer visual attributes from real images onto synthetic 3D surfaces. Traditional methods struggle with resolution mismatch and the inherent discreteness of point correspondences. In contrast, resolution-robust functional maps enable smooth attribute propagation but rely on near-isometry assumptions and topological consistency. To address these limitations, we propose RealSkin, a self-supervised framework that performs correspondence optimization in a learned spectral domain, guided by spatial correspondences. We first introduce a spatial-guided registration algorithm to establish coarse correspondences under severe topological discrepancies. To relax strict isometric assumptions and handle partial correspondences, we further design a spectral-aware neural adjoint network that incorporates partial correspondences into a neural function space and models non-isometric residuals for correspondence refinement. Experimental results demonstrate that our method achieves state-of-the-art performance on challenging real-to-synthetic scenarios. The code will be publicly released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。