arXiv:2605.01450cs.CV2026-05被引 3

无需注册数据训练,实现高精度3D人脸重建。

Registration-Free Learnable Multi-View Capture of Faces in Dense Semantic Correspondence

论文配图:Registration-Free Learnable Multi-View Capture of Faces in Dense Semantic Correspondence
图 1 · 摘自论文原文
  • 用伪线性逆运动学保证拓扑一致性,摆脱对注册数据依赖。
  • 采用点图与法向损失,提升重建稳定性和视觉质量。
  • 测试时优化可融合前馈效率与迭代精度,适合高质量重建场景。

近期方法如ToFu和TEMPEH通过从标定多视角图像中直接预测稠密语义对应关系的3D网格,提供了传统注册流程的自动化替代方案。然而这些学习方法仍需依赖其试图取代的慢速人工注册流程作为训练监督。我们提出MOCHI(Multi-view Optimizable Correspondence of Heads from Images),一种无需注册训练数据的多视角3D人脸预测框架。MOCHI通过伪线性逆运动学求解器强制拓扑一致性,消除注册数据依赖;语义对齐由仅在合成数据上训练的2D关键点检测器提供。分析发现,标准点到面距离在无注册设置下导致训练不稳和视觉伪影,因此我们引入基于点图和法向的损失,提供更平滑梯度与更高重建保真度。最后,我们设计了测试时优化方案,在数十次迭代内微调网络权重,兼顾前馈效率与迭代精度,使MOCHI在重建准确率和视觉质量上超越传统人工流水线。代码与模型已公开:https://filby89.github.io/mochi。

原文摘要 · Abstract (English)

Recent frameworks like ToFu and TEMPEH provide an automated alternative to classical registration pipelines by predicting 3D meshes in dense semantic correspondence directly from calibrated multi-view images. However, these learning-based methods rely on the slow, manual registration pipelines they aim to replace for their training supervision. We overcome this limitation with MOCHI (Multi-view Optimizable Correspondence of Heads from Images), a multi-view 3D face prediction framework trained without requiring registered training data. MOCHI eliminates the registration data dependency by enforcing topological consistency through a pseudo-linear inverse kinematic solver. Semantic alignment is guided by dense keypoints from a 2D landmark predictor trained exclusively on synthetic data. Our analysis further reveals that standard point-to-surface distances induce training instabilities and visual artifacts in registration-free settings. We propose pointmap- and normal-based losses instead, which provide smoother gradients and superior reconstruction fidelity. Finally, we introduce a test-time optimization scheme that refines network weights over a few dozen iterations. This approach bridges the gap between feed-forward efficiency and iterative optimization precision, allowing MOCHI to outperform traditional labor-intensive pipelines in both reconstruction accuracy and visual quality. Code and model are public at: https://filby89.github.io/mochi.

3D人脸多视角重建无注册训练测试时优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。