arXiv:2412.05961cs.CV2024-12被引 1

单图实时重建高精度人体3D模型,速度与质量兼得。

FOF-X: Towards Real-time Detailed Human Reconstruction from a Single Image

  • 用傅里叶占用场压缩3D表示,兼容2D卷积网络
  • 在多种数据集上实现实时重建,优于现有方法
  • 适合需要快速高质量人体建模的场景

我们提出FOF-X,实现从单张图像中实时重建高细节人体几何。现有3D表示计算开销大,难以兼顾实时性与高质量。为此,我们设计傅里叶占用场(FOF),通过学习傅里叶级数来高效表达3D占据场。其核心是将3D占据场分解为2D向量场,保留3D拓扑结构与空间关系,同时支持2D卷积神经网络处理。该表示打通3D与2D域的鸿沟,可融入人体参数化模型作为先验,提升重建鲁棒性。基于FOF,我们构建新框架FOF-X,有效缓解纹理与光照导致的性能下降,增强对训练图像与真实图像间域差距的适应能力。此外,我们在FOF与网格表示间引入拉普拉斯约束与基于自动机的不连续匹配算法,进一步提升重建质量和鲁棒性。在多个数据集和真实采集数据上验证表明,FOF-X达到当前最优性能。代码已开源:https://cic.tju.edu.cn/faculty/likun/projects/FOFX/index.html。

原文摘要 · Abstract (English)

We introduce FOF-X for real-time reconstruction of detailed human geometry from a single image. Balancing real-time speed against high-quality results is a persistent challenge, mainly due to the high computational demands of existing 3D representations. To address this, we propose Fourier Occupancy Field (FOF), an efficient 3D representation by learning the Fourier series. The core of FOF is to factorize a 3D occupancy field into a 2D vector field, retaining topology and spatial relationships within the 3D domain while facilitating compatibility with 2D convolutional neural networks. Such a representation bridges the gap between 3D and 2D domains, enabling the integration of human parametric models as priors and enhancing the reconstruction robustness. Based on FOF, we design a new reconstruction framework, FOF-X, to avoid the performance degradation caused by texture and lighting. This enables our real-time reconstruction system to better handle the domain gap between training images and real images. Additionally, in FOF-X, we enhance the inter-conversion algorithms between FOF and mesh representations with a Laplacian constraint and an automaton-based discontinuity matcher, improving both quality and robustness. We validate the strengths of our approach on different datasets and real-captured data, where FOF-X achieves new state-of-the-art results. The code has already been released for research purposes at https://cic.tju.edu.cn/faculty/likun/projects/FOFX/index.html.

3D重建实时渲染傅里叶表示

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。