arXiv:2505.00421cs.CV2025-05

用单目视频实时重建可动画化且细节丰富的2D高斯人像

Real-Time Animatable 2DGS-Avatars with Detail Enhancement from Monocular Videos

  • 基于2D高斯溅射与全局SMPL姿态参数,实现精准对齐与自然动画
  • 引入旋转补偿网络,显著提升非刚性形变处理能力,动画无伪影
  • 适合游戏、AR和社交应用,实现在单目视频下高质量人像重建

从单目视频中高保真、可动画化地重建3D人体形象,具有降低对复杂硬件依赖的潜力,广泛适用于游戏开发、增强现实和社交媒体。然而,现有方法在捕捉精细几何细节和保持动画稳定性方面仍面临挑战,尤其在动态或复杂姿态下表现不佳。为此,我们提出一种基于2D高斯溅射(2DGS)的新型实时可动画化人体重建框架。通过结合2DGS与全局SMPL姿态参数,该框架有效校正位置与旋转偏差,实现稳定自然的姿势驱动动画。此外,我们设计了旋转补偿网络(RCN),通过融合局部几何特征与全局姿态参数来学习旋转残差,显著改善非刚性形变处理能力,并确保动画过程中的平滑过渡,无明显伪影。实验结果表明,该方法能从单目视频中成功重建出真实感强、高度可动画化的3D人体形象,在保留细粒度细节的同时保证姿态变化的稳定性与自然性。在公开基准测试中,本方法在重建质量与动画鲁棒性方面均优于当前最先进水平。

原文摘要 · Abstract (English)

High-quality, animatable 3D human avatar reconstruction from monocular videos offers significant potential for reducing reliance on complex hardware, making it highly practical for applications in game development, augmented reality, and social media. However, existing methods still face substantial challenges in capturing fine geometric details and maintaining animation stability, particularly under dynamic or complex poses. To address these issues, we propose a novel real-time framework for animatable human avatar reconstruction based on 2D Gaussian Splatting (2DGS). By leveraging 2DGS and global SMPL pose parameters, our framework not only aligns positional and rotational discrepancies but also enables robust and natural pose-driven animation of the reconstructed avatars. Furthermore, we introduce a Rotation Compensation Network (RCN) that learns rotation residuals by integrating local geometric features with global pose parameters. This network significantly improves the handling of non-rigid deformations and ensures smooth, artifact-free pose transitions during animation. Experimental results demonstrate that our method successfully reconstructs realistic and highly animatable human avatars from monocular videos, effectively preserving fine-grained details while ensuring stable and natural pose variation. Our approach surpasses current state-of-the-art methods in both reconstruction quality and animation robustness on public benchmarks.

3D重建2D高斯动画生成单目视频

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。