用合成数据训练的先验模型，让单摄像头视频也能生成360度可渲染的高质虚拟人。
GASP: Gaussian Avatars with Synthetic Priors
- 用合成数据构建高保真先验，指导单图/视频的高质重建
- 仅需单摄像头数据即可实现360度自由视角渲染，支持70fps实时播放
- 先验仅用于训练阶段，推理无负担，适合消费级硬件部署
高斯点阵（Gaussian Splatting）已革新实时照片级渲染。其最受欢迎的应用之一是创建可动画化的虚拟人，即高斯虚拟人。现有方法在质量和渲染效率上不断突破，但仍存在两大局限：要么依赖昂贵的多相机阵列实现自由视角，要么仅能从固定视角高质量渲染。理想方案应仅需短时单目视频或图像（如网络摄像头），即可生成任意视角可见的高质虚拟人。为此，我们提出GASP：基于合成先验的高斯虚拟人。通过利用合成数据的像素级精确特性，训练一个高斯虚拟人先验模型。将该先验拟合至单张照片或视频并微调后，即可获得支持360°渲染的高质量高斯虚拟人。该先验仅用于训练阶段，不参与推理，保障实时性。实验表明，本方法仅需有限数据即可生成可动画、可实时渲染的虚拟人，在商用硬件上实现70帧/秒的流畅表现。
原文摘要 · Abstract (English)
Gaussian Splatting has changed the game for real-time photo-realistic rendering. One of the most popular applications of Gaussian Splatting is to create animatable avatars, known as Gaussian Avatars. Recent works have pushed the boundaries of quality and rendering efficiency but suffer from two main limitations. Either they require expensive multi-camera rigs to produce avatars with free-view rendering, or they can be trained with a single camera but only rendered at high quality from this fixed viewpoint. An ideal model would be trained using a short monocular video or image from available hardware, such as a webcam, and rendered from any view. To this end, we propose GASP: Gaussian Avatars with Synthetic Priors. To overcome the limitations of existing datasets, we exploit the pixel-perfect nature of synthetic data to train a Gaussian Avatar prior. By fitting this prior model to a single photo or video and fine-tuning it, we get a high-quality Gaussian Avatar, which supports 360$^\circ$ rendering. Our prior is only required for fitting, not inference, enabling real-time application. Through our method, we obtain high-quality, animatable Avatars from limited data which can be animated and rendered at 70fps on commercial hardware. See our project page (https://microsoft.github.io/GASP/) for results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。