用软绑定和时序密度控制,让单目视频生成更自然的3D人脸动画。
STAvatar: Soft Binding and Temporal Density Control for Monocular 3D Head Avatars Reconstruction
- 通过UV空间软绑定学习高斯点特征偏移,实现动态重采样
- 基于时序聚类与融合感知误差,精准提升遮挡区域细节
- 适合需要高保真人脸重建的虚拟人、影视特效应用
从单目视频重建高保真且可动画化的3D人脸仍是挑战性任务。现有基于3D高斯溅射的方法通常将高斯点绑定到网格三角面片,并仅使用线性混合皮肤化建模形变,导致运动僵硬且表达力有限。此外,缺乏针对频繁遮挡区域(如嘴内部、眼睑)的专门处理策略。为此,我们提出STAvatar,包含两个核心组件:(1) 基于UV自适应的软绑定框架,结合图像与几何先验,在UV空间中学习每个高斯点的特征偏移,支持动态重采样,与自适应密度控制(ADC)完全兼容,并增强对形状与纹理变化的适应性;(2) 时序自适应密度控制策略,首先对结构相似帧进行聚类,以更精准计算致密化标准;进一步引入一种新型融合感知误差作为致密化判据,联合捕捉几何与纹理差异,促进在需精细细节区域的致密化。在四个基准数据集上的大量实验表明,STAvatar在捕捉细粒度细节及重建频繁遮挡区域方面达到当前最优性能。
原文摘要 · Abstract (English)
Reconstructing high-fidelity and animatable 3D head avatars from monocular videos remains a challenging yet essential task. Existing methods based on 3D Gaussian Splatting typically bind Gaussians to mesh triangles and model deformations solely via Linear Blend Skinning, which results in rigid motion and limited expressiveness. Moreover, they lack specialized strategies to handle frequently occluded regions (e.g., mouth interiors, eyelids). To address these limitations, we propose STAvatar, which consists of two key components: (1) a UV-Adaptive Soft Binding framework that leverages both image-based and geometric priors to learn per-Gaussian feature offsets within the UV space. This UV representation supports dynamic resampling, ensuring full compatibility with Adaptive Density Control (ADC) and enhanced adaptability to shape and textural variations. (2) a Temporal ADC strategy, which first clusters structurally similar frames to facilitate more targeted computation of the densification criterion. It further introduces a novel fused perceptual error as clone criterion to jointly capture geometric and textural discrepancies, encouraging densification in regions requiring finer details. Extensive experiments on four benchmark datasets demonstrate that STAvatar achieves state-of-the-art reconstruction performance, especially in capturing fine-grained details and reconstructing frequently occluded regions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。