通过智能剔除空域采样点,实现高效3D虚拟人生成。
EPSilon: Efficient Point Sampling for Lightening of Hybrid-based 3D Avatar Generation
- 提出空射线与空区间剔除策略,跳过无效采样点。
- 仅用3.9%采样点保持画质,推理速度提升20倍。
- 适合需要快速生成高质量虚拟人的应用开发。
神经辐射场(NeRF)的快速发展推动了单目视频生成可动画化虚拟人。然而,纯NeRF缺乏细节,因此出现结合SMPL网格与NeRF的混合表示。尽管此类方法生成逼真,但推理极慢——因需对每个采样点进行基于SMPL皮肤权重的形变计算。我们发现多数采样点位于空域,不影响质量却增加延迟。为此,提出EPSilon:引入空射线剔除(ERO)和空区间剔除(EIO),分别移除穿过空域的射线及未被衣物或网格占据的采样区间。该精细采样策略在减少形变计算开销的同时,精准定位关键区域,支持单阶段NeRF结构而无需分层采样。相比现有方法,EPSilon在仅使用3.9%采样点的前提下保持生成质量,推理速度提升约20倍,训练收敛速度加快4倍。视频结果见https://github.com/seungjun-moon/epsilon。
原文摘要 · Abstract (English)
The rapid advancement of neural radiance fields (NeRF) has paved the way to generate animatable human avatars from a monocular video. However, the sole usage of NeRF suffers from a lack of details, which results in the emergence of hybrid representation that utilizes SMPL-based mesh together with NeRF representation. While hybrid-based models show photo-realistic human avatar generation qualities, they suffer from extremely slow inference due to their deformation scheme: to be aligned with the mesh, hybrid-based models use the deformation based on SMPL skinning weights, which needs high computational costs on each sampled point. We observe that since most of the sampled points are located in empty space, they do not affect the generation quality but result in inference latency with deformation. In light of this observation, we propose EPSilon, a hybrid-based 3D avatar generation scheme with novel efficient point sampling strategies that boost both training and inference. In EPSilon, we propose two methods to omit empty points at rendering; empty ray omission (ERO) and empty interval omission (EIO). In ERO, we wipe out rays that progress through the empty space. Then, EIO narrows down the sampling interval on the ray, which wipes out the region not occupied by either clothes or mesh. The delicate sampling scheme of EPSilon enables not only great computational cost reduction during deformation but also the designation of the important regions to be sampled, which enables a single-stage NeRF structure without hierarchical sampling. Compared to existing methods, EPSilon maintains the generation quality while using only 3.9% of sampled points and achieves around 20 times faster inference, together with 4 times faster training convergence. We provide video results on https://github.com/seungjun-moon/epsilon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。