用高斯点增强人体虚拟人渲染，实时生成逼真可动形象
GaussianGAN: Real-Time Photorealistic controllable Human Avatars
- 基于骨骼结构的新型高斯点稠密化策略构建人体表面
- 实测79帧/秒，视觉质量在两个数据集上分别达32.94db和33.39db
- 适合需要实时高保真虚拟人生成的交互应用
得益于神经渲染的快速发展，逼真且可控制的人体虚拟人受到研究界广泛关注，提供了快速而真实的合成工具。然而，现有方法存在明显模糊问题。为此，我们提出GaussianGAN，一种用于实时逼真人物渲染的可动画虚拟人方法。通过从估计骨骼肢体周围圆柱面构建高斯点，引入新颖的高斯点稠密化策略。结合相机标定，利用新型视图分割模块生成精确语义分割图。最后，采用UNet生成器结合渲染的高斯点特征与分割图，生成逼真数字虚拟人。该方法实现79帧/秒的实时渲染，视觉感知与质量优于以往方法，在ZJU Mocap数据集上达到32.94db像素保真度，Thuman4数据集上达33.39db。
原文摘要 · Abstract (English)
Photorealistic and controllable human avatars have gained popularity in the research community thanks to rapid advances in neural rendering, providing fast and realistic synthesis tools. However, a limitation of current solutions is the presence of noticeable blurring. To solve this problem, we propose GaussianGAN, an animatable avatar approach developed for photorealistic rendering of people in real-time. We introduce a novel Gaussian splatting densification strategy to build Gaussian points from the surface of cylindrical structures around estimated skeletal limbs. Given the camera calibration, we render an accurate semantic segmentation with our novel view segmentation module. Finally, a UNet generator uses the rendered Gaussian splatting features and the segmentation maps to create photorealistic digital avatars. Our method runs in real-time with a rendering speed of 79 FPS. It outperforms previous methods regarding visual perception and quality, achieving a state-of-the-art results in terms of a pixel fidelity of 32.94db on the ZJU Mocap dataset and 33.39db on the Thuman4 dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。