用混合表面与体素高斯表示,实现高保真头像重建与实时外观编辑。
SVG-Head: Hybrid Surface-Volumetric Gaussians for High-Fidelity Head Reconstruction and Real-Time Editing
- 表面高斯+体素高斯混合建模,几何与外观显式分离
- 首次生成可实时编辑的显式纹理图,重建质量优异
- 适用于AR/VR中的头像实时定制,支持快速渲染
高保真可编辑头像在计算机视觉与图形学中至关重要,推动了诸多AR/VR应用的发展。尽管近期进展已实现逼真的渲染与合理动画,但头像编辑(尤其是实时外观编辑)仍因隐式表示和几何与全局外观耦合建模而困难。为此,我们提出表面-体素高斯头像(SVG-Head),一种新型混合表示:以FLAME网格为约束,用3D高斯显式建模几何,并通过解耦纹理图捕捉全局外观。该方法包含两类高斯:表面高斯使用可学习纹理图显式建模头像外观,支持实时纹理编辑;体素高斯提升非朗伯区域(如嘴唇、头发)的重建质量。为建立3D世界与纹理空间的对应关系,提出基于网格的高斯UV映射方法,利用FLAME网格的UV坐标获得清晰纹理图并实现实时渲染。进一步设计分层优化策略,在重建质量与编辑灵活性间取得最优平衡。在NeRSemble数据集上的实验表明,SVG-Head不仅生成高保真渲染结果,更是首个为高斯头像生成显式纹理图并支持实时外观编辑的方法。
原文摘要 · Abstract (English)
Creating high-fidelity and editable head avatars is a pivotal challenge in computer vision and graphics, boosting many AR/VR applications. While recent advancements have achieved photorealistic renderings and plausible animation, head editing, especially real-time appearance editing, remains challenging due to the implicit representation and entangled modeling of the geometry and global appearance. To address this, we propose Surface-Volumetric Gaussian Head Avatar (SVG-Head), a novel hybrid representation that explicitly models the geometry with 3D Gaussians bound on a FLAME mesh and leverages disentangled texture images to capture the global appearance. Technically, it contains two types of Gaussians, in which surface Gaussians explicitly model the appearance of head avatars using learnable texture images, facilitating real-time texture editing, while volumetric Gaussians enhance the reconstruction quality of non-Lambertian regions (e.g., lips and hair). To model the correspondence between 3D world and texture space, we provide a mesh-aware Gaussian UV mapping method, which leverages UV coordinates given by the FLAME mesh to obtain sharp texture images and real-time rendering speed. A hierarchical optimization strategy is further designed to pursue the optimal performance in both reconstruction quality and editing flexibility. Experiments on the NeRSemble dataset show that SVG-Head not only generates high-fidelity rendering results, but also is the first method to obtain explicit texture images for Gaussian head avatars and support real-time appearance editing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。