arXiv:2502.09617cs.CV2025-02ICLR被引 11

用多分辨率高斯网格实现快速通用人体渲染,1秒内完成重建。

LIFe-GoM: Generalizable Human Rendering with Learned Iterative Feedback Over Multi-Resolution Gaussians-on-Mesh

  • 引入迭代反馈机制,单次推理中逐步优化人体形状表示。
  • 在1024×1024分辨率下实现95.1帧率,重建耗时小于1秒。
  • 适合需要快速高质量人体渲染的虚拟人应用。

从稀疏输入中实现可泛化的可驱动人体渲染,依赖于大规模数据中提取的数据先验和归纳偏置,以避免场景特定优化并实现快速重建。这带来两大挑战:其一,与场景特定优化中的迭代梯度调整不同,泛化方法必须在推理时单次完成人体形状表示重建;其二,渲染需兼顾计算效率与高分辨率。为此,我们对近期提出的双形状表示(结合网格与高斯点)进行两方面改进:为提升重建质量,提出一种迭代反馈更新框架,在重建过程中逐次优化初始人体形状;为实现高效且高分辨率渲染,研究了一种耦合多分辨率高斯网格表示。我们在THuman2.0、XHuman和AIST++等挑战性数据集上进行了评估。所提方法可在1秒内完成稀疏输入下的可驱动表示重建,1024×1024分辨率下渲染达95.1帧/秒,且在THuman2.0上取得PSNR 24.65、LPIPS 110.82、FID 51.27,优于当前最先进水平。

原文摘要 · Abstract (English)

Generalizable rendering of an animatable human avatar from sparse inputs relies on data priors and inductive biases extracted from training on large data to avoid scene-specific optimization and to enable fast reconstruction. This raises two main challenges: First, unlike iterative gradient-based adjustment in scene-specific optimization, generalizable methods must reconstruct the human shape representation in a single pass at inference time. Second, rendering is preferably computationally efficient yet of high resolution. To address both challenges we augment the recently proposed dual shape representation, which combines the benefits of a mesh and Gaussian points, in two ways. To improve reconstruction, we propose an iterative feedback update framework, which successively improves the canonical human shape representation during reconstruction. To achieve computationally efficient yet high-resolution rendering, we study a coupled-multi-resolution Gaussians-on-Mesh representation. We evaluate the proposed approach on the challenging THuman2.0, XHuman and AIST++ data. Our approach reconstructs an animatable representation from sparse inputs in less than 1s, renders views with 95.1FPS at $1024 \times 1024$, and achieves PSNR/LPIPS*/FID of 24.65/110.82/51.27 on THuman2.0, outperforming the state-of-the-art in rendering quality.

人体渲染高斯网格实时渲染可泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。