用可学习的样条体表示人脸,支持高精度重建与局部编辑。
Representing 3D Faces with Learnable B-Spline Volumes
- 以高维控制特征替代传统3D控制点,提升几何表达能力。
- 在8×8×8网格上实现端到端3D人脸重建,比现有方法更精确。
- 适合需要精细建模与局部修改的3D人脸应用,如动画与医疗建模。
我们提出CUBE(基于控制的统一样条编码),一种结合样条体与学习特征的人脸几何表示方法,用于3D扫描配准和单目3D人脸重建。与传统的3D控制点样条不同,CUBE采用8×8×8的高维控制特征格网,显著增强模型表达能力。该方法通过两阶段映射:先用样条基局部融合控制特征生成高维特征向量,其前三个分量定义基础网格;再通过小型MLP预测从基础形状的残差位移,得到最终3D坐标。为实现密集语义对应,对固定模板网格采样点进行查询。关键优势是保留传统样条的局部支撑性,可通过更新单个控制特征实现局部表面编辑。我们训练基于Transformer的编码器,从无结构点云和单目图像预测CUBE控制特征,在扫描配准任务中达到当前最优性能。
原文摘要 · Abstract (English)
We present CUBE (Control-based Unified B-spline Encoding), a new geometric representation for human faces that combines B-spline volumes with learned features, and demonstrate its use as a decoder for 3D scan registration and monocular 3D face reconstruction. Unlike existing B-spline representations with 3D control points, CUBE is parametrized by a lattice (e.g., 8 x 8 x 8) of high-dimensional control features, increasing the model's expressivity. These features define a continuous, two-stage mapping from a 3D parametric domain to 3D Euclidean space via an intermediate feature space. First, high-dimensional control features are locally blended using the B-spline bases, yielding a high-dimensional feature vector whose first three values define a 3D base mesh. A small MLP then processes this feature vector to predict a residual displacement from the base shape, yielding the final refined 3D coordinates. To reconstruct 3D surfaces in dense semantic correspondence, CUBE is queried at 3D coordinates sampled from a fixed template mesh. Crucially, CUBE retains the local support property of traditional B-spline representations, enabling local surface editing by updating individual control features. We demonstrate the strengths of this representation by training transformer-based encoders to predict CUBE's control features from unstructured point clouds and monocular images, achieving state-of-the-art scan registration results compared to recent baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。