让虚拟人头像与手部互动更真实，支持高分辨率渲染。
NBAvatar: Neural Billboards Avatars with Realistic Hand-Face Interaction
- 用显式平面基元+神经渲染结合建模，保持动作连贯性。
- 高分辨率下比高斯方法降低30% LPIPS，PSNR和SSIM均提升。
- 适合需要逼真手脸交互的虚拟形象应用，如游戏、元宇宙。
我们提出NBAvatar——一种用于真实渲染头部虚拟形象的方法，可处理手与面部交互引起的非刚性形变。通过将定向平面基元训练与神经渲染相结合，构建新型动画虚拟形象表示。该显式与隐式表示的融合使NBAvatar能保持时序与姿态一致性，并利用神经渲染提供精细外观细节。实验表明，NBAvatar能隐式学习面部-手部交互导致的颜色变化，在新视角与新姿态渲染质量上超越现有方法。具体而言，在高分辨率兆像素渲染下，相比基于高斯的虚拟形象方法，其LPIPS降低最多达30%，同时提升PSNR和SSIM；在结构相似性方面，优于当前最先进的手脸交互方法InteractAvatar。
原文摘要 · Abstract (English)
We present NBAvatar - a method for realistic rendering of head avatars handling non-rigid deformations caused by hand-face interaction. We introduce a novel representation for animated avatars by combining the training of oriented planar primitives with neural rendering. Such a combination of explicit and implicit representations enables NBAvatar to handle temporally and pose-consistent geometry, along with fine-grained appearance details provided by the neural rendering technique. In our experiments, we demonstrate that NBAvatar implicitly learns color transformations caused by face-hand interactions and surpasses existing approaches in terms of novel-view and novel-pose rendering quality. Specifically, NBAvatar achieves up to 30% LPIPS reduction under high-resolution megapixel rendering compared to Gaussian-based avatar methods, while also improving PSNR and SSIM, and achieves higher structural similarity compared to the state-of-the-art hand-face interaction method InteractAvatar.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。