用时间码本记录与检索，让虚拟人像既逼真又可动画。
R3-Avatar: Record and Retrieve Temporal Codebook for Reconstructing Photorealistic Human Avatars
- 记录不同时间点的外观变化，构建时间码本。
- 在新姿态下检索最相似训练姿态，提升渲染质量。
- 适合需要高保真且灵活动画的虚拟人场景。
我们提出R3-Avatar，引入时间码本，解决现有3D人体虚拟人像难以同时实现高质量渲染与良好动画性的难题。当前基于视频的重建方法或只关注渲染质量而缺乏动画支持，或通过姿态-外观映射进行动画，但在训练姿态有限或衣物复杂时性能下降。本文采用“记录-检索-重建”策略：通过去歧义的时间戳将外观随时间的变化存入码本，确保新视角下高质量渲染；在新姿态下，通过匹配最相似的训练姿态检索对应时间码本，增强外观表现。R3-Avatar在极端场景（如训练姿态少、衣物复杂）下显著优于前沿方法，有效缓解视觉质量退化问题。
原文摘要 · Abstract (English)
We present R3-Avatar, incorporating a temporal codebook, to overcome the inability of human avatars to be both animatable and of high-fidelity rendering quality. Existing video-based reconstruction of 3D human avatars either focuses solely on rendering, lacking animation support, or learns a pose-appearance mapping for animating, which degrades under limited training poses or complex clothing. In this paper, we adopt a "record-retrieve-reconstruct" strategy that ensures high-quality rendering from novel views while mitigating degradation in novel poses. Specifically, disambiguating timestamps record temporal appearance variations in a codebook, ensuring high-fidelity novel-view rendering, while novel poses retrieve corresponding timestamps by matching the most similar training poses for augmented appearance. Our R3-Avatar outperforms cutting-edge video-based human avatar reconstruction, particularly in overcoming visual quality degradation in extreme scenarios with limited training human poses and complex clothing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。