arXiv:2412.00845cs.CVcs.GR2024-12被引 1

用对齐网格的高斯表示,实现单视频驱动的可动画真人虚拟形象。

SAGA: Surface-Aligned Gaussian Avatar

  • 高斯点先贴合网格再松脱,兼顾几何精度与表达力。
  • 在新视角和新姿态下生成更稳定、无噪声的动态图像。
  • 适合做实时虚拟人动画或数字孪生场景的开发者使用。

本文提出SAGA——一种用于从单目视频创建可动画真人虚拟形象的表面对齐高斯表示方法,旨在提升新视角与新姿态合成性能的同时,确保快速训练与实时渲染。近年来,3DGS已成为比NeRF更高效、更具表现力的替代方案,并被用于构建动态人体虚拟形象。然而,在严重病态的单目动态重建任务中,高斯点容易对不断变化的区域(如衣物褶皱或阴影)过拟合,这些区域缺乏一致监督,导致几何噪声和突变形变,难以泛化至新视角和新姿态。为解决这一问题,本文提出SAGA,即表面对齐高斯虚拟形象,通过将高斯点与网格对齐,强制几何清晰且形变一致,从而提升泛化能力。不同于现有严格绑定方法带来的表达力受限与真实感不足,SAGA采用两阶段对齐策略:第一阶段允许高斯点在网格上流动,提升灵活性;第二阶段引入高斯-网格对齐正则化,使高斯点脱离网格后仍保持位置与朝向与对应三角面片一致,释放表达潜力并维持几何对齐。此外,针对优化过程中高斯点可能漂出边界三角面片的问题,提出高效的‘走动于网格’策略,动态更新边界三角面片。

原文摘要 · Abstract (English)

This paper presents a Surface-Aligned Gaussian representation for creating animatable human avatars from monocular videos,aiming at improving the novel view and pose synthesis performance while ensuring fast training and real-time rendering. Recently,3DGS has emerged as a more efficient and expressive alternative to NeRF, and has been used for creating dynamic human avatars. However,when applied to the severely ill-posed task of monocular dynamic reconstruction, the Gaussians tend to overfit the constantly changing regions such as clothes wrinkles or shadows since these regions cannot provide consistent supervision, resulting in noisy geometry and abrupt deformation that typically fail to generalize under novel views and poses.To address these limitations, we present SAGA,i.e.,Surface-Aligned Gaussian Avatar,which aligns the Gaussians with a mesh to enforce well-defined geometry and consistent deformation, thereby improving generalization under novel views and poses. Unlike existing strict alignment methods that suffer from limited expressive power and low realism,SAGA employs a two-stage alignment strategy where the Gaussians are first adhered on while then detached from the mesh, thus facilitating both good geometry and high expressivity. In the Adhered Stage, we improve the flexibility of Adhered-on-Mesh Gaussians by allowing them to flow on the mesh, in contrast to existing methods that rigidly bind Gaussians to fixed location. In the second Detached Stage, we introduce a Gaussian-Mesh Alignment regularization, which allows us to unleash the expressivity by detaching the Gaussians but maintain the geometric alignment by minimizing their location and orientation offsets from the bound triangles. Finally, since the Gaussians may drift outside the bound triangles during optimization, an efficient Walking-on-Mesh strategy is proposed to dynamically update the bound triangles.

三维重建虚拟人高斯溅射

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。