arXiv:2410.08840cs.CV2024-10NeurIPS被引 6

仅用一张图生成可交互的手部3D动画模型,效果超越现有方法。

Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatars

  • 分两阶段建模:优化身份图+学习隐式特征,适应不同手形
  • 在交互区域使用注意力与自适应优化,提升渲染质量
  • 适合需要快速生成手部虚拟形象的AR/VR应用

本文提出一种基于3D高斯点云(GS)的单图像手部虚拟人生成方法,旨在构建可动画化的交互手部模型。现有基于GS的方法因输入视角有限、手部姿态多样及遮挡问题,常产生较差结果。为此,我们设计了两阶段交互感知框架,利用跨主体手部先验,在交互区域对3D高斯进行精细化优化。为应对手部差异,将3D表示解耦为优化型身份图与学习型隐式几何特征和神经纹理图;前者由训练网络捕捉姿态、形状、纹理先验,后者实现对分布外手部的高效单图拟合。进一步引入交互感知注意力模块与自适应高斯优化模块,显著提升手内及手间交互区域的图像渲染质量。在大规模InterHand2.6M数据集上的大量实验表明,该方法显著优于当前最佳性能。

原文摘要 · Abstract (English)

In this paper, we propose to create animatable avatars for interacting hands with 3D Gaussian Splatting (GS) and single-image inputs. Existing GS-based methods designed for single subjects often yield unsatisfactory results due to limited input views, various hand poses, and occlusions. To address these challenges, we introduce a novel two-stage interaction-aware GS framework that exploits cross-subject hand priors and refines 3D Gaussians in interacting areas. Particularly, to handle hand variations, we disentangle the 3D presentation of hands into optimization-based identity maps and learning-based latent geometric features and neural texture maps. Learning-based features are captured by trained networks to provide reliable priors for poses, shapes, and textures, while optimization-based identity maps enable efficient one-shot fitting of out-of-distribution hands. Furthermore, we devise an interaction-aware attention module and a self-adaptive Gaussian refinement module. These modules enhance image rendering quality in areas with intra- and inter-hand interactions, overcoming the limitations of existing GS-based methods. Our proposed method is validated via extensive experiments on the large-scale InterHand2.6M dataset, and it significantly improves the state-of-the-art performance in image quality. Project Page: \url{https://github.com/XuanHuang0/GuassianHand}.

3D高斯手部建模单图像交互感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。