用分层运动上下文提升3D高斯人体渲染细节与速度
Sequential Gaussian Avatars with Hierarchical Motion Context
- 分步建模骨架与顶点级运动，捕捉复杂姿态下的外观变化
- 在多个时空尺度上采样，显著提升运动条件的鲁棒性
- 比最新时序NeRF快数个量级，效果更优或相当
神经渲染技术推动了3D人体化身的画质进步，其中3DGS方法实现了实时渲染。然而，基于SMPL的3DGS人体模型在拟合过程中因姿态到外观映射复杂，仍难以捕捉精细外观细节。本文提出SeqAvatar，通过挖掘显式3DGS表示，基于分层运动上下文更精准建模人体化身。具体而言，采用从粗到细的运动条件，同时融合整体骨骼运动与细粒度顶点运动以实现非刚性形变建模。为增强运动条件的鲁棒性，引入时空多尺度采样策略，分层整合更多运动线索。大量实验表明，该方法显著优于现有3DGS基模型，在渲染速度上比最新融合时序信息的NeRF模型快数个量级，且性能至少相当甚至更优。
原文摘要 · Abstract (English)
The emergence of neural rendering has significantly advanced the rendering quality of 3D human avatars, with the recently popular 3DGS technique enabling real-time performance. However, SMPL-driven 3DGS human avatars still struggle to capture fine appearance details due to the complex mapping from pose to appearance during fitting. In this paper, we propose SeqAvatar, which excavates the explicit 3DGS representation to better model human avatars based on a hierarchical motion context. Specifically, we utilize a coarse-to-fine motion conditions that incorporate both the overall human skeleton and fine-grained vertex motions for non-rigid deformation. To enhance the robustness of the proposed motion conditions, we adopt a spatio-temporal multi-scale sampling strategy to hierarchically integrate more motion clues to model human avatars. Extensive experiments demonstrate that our method significantly outperforms 3DGS-based approaches and renders human avatars orders of magnitude faster than the latest NeRF-based models that incorporate temporal context, all while delivering performance that is at least comparable or even superior. Project page: https://zezeaaa.github.io/projects/SeqAvatar/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。