无需网格，仅用骨骼即可高保真渲染手部，支持复杂交互
HandOcc: NeRF-based Hand Rendering with Occupancy Networks
- 基于占用表示的NeRF框架，摆脱传统参数化网格束缚
- 仅需3D骨骼输入，在InterHand2.6M上达到最优性能
- 适用于无现成模型的手部，适合虚拟角色与动作迁移
我们提出HandOcc，一种基于占用表示的新型手部渲染框架。主流渲染方法如NeRF常结合参数化网格以实现可变形手模型，但此类方法在网格保真度与参数模型复杂度间存在权衡。参数化网格结构虽简洁，却受限于初始网格，难以推广至无对应参数模型的对象，且估计结果依赖网格分辨率与拟合精度。本文提出无网格3D渲染流程，应用于手部建模。仅需输入3D骨骼，通过卷积模型提取目标外观，利用条件于占用表示的NeRF渲染器实现。该方法借助手部占用信息增强手与手之间的交互建模，提升渲染质量，支持快速生成与优秀外观迁移效果。在InterHand2.6M基准数据集上，取得当前最优结果。
原文摘要 · Abstract (English)
We propose HandOcc, a novel framework for hand rendering based upon occupancy. Popular rendering methods such as NeRF are often combined with parametric meshes to provide deformable hand models. However, in doing so, such approaches present a trade-off between the fidelity of the mesh and the complexity and dimensionality of the parametric model. The simplicity of parametric mesh structures is appealing, but the underlying issue is that it binds methods to mesh initialization, making it unable to generalize to objects where a parametric model does not exist. It also means that estimation is tied to mesh resolution and the accuracy of mesh fitting. This paper presents a pipeline for meshless 3D rendering, which we apply to the hands. By providing only a 3D skeleton, the desired appearance is extracted via a convolutional model. We do this by exploiting a NeRF renderer conditioned upon an occupancy-based representation. The approach uses the hand occupancy to resolve hand-to-hand interactions further improving results, allowing fast rendering, and excellent hand appearance transfer. On the benchmark InterHand2.6M dataset, we achieved state-of-the-art results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。