从单目视频重建松垮衣物下的人体,精度远超现有方法。
ReLoo: Reconstructing Humans Dressed in Loose Garments from Monocular Video in the Wild
- 分层神经人体建模,将衣物与身体解耦表示。
- 虚拟骨骼自由变形,精准捕捉衣物非刚性形变。
- 适合在真实场景中穿宽松衣服的人体重建任务。
尽管近年来单目视频中人体三维重建取得显著进展,但现有先进方法难以处理穿着松垮衣物时产生的大范围非刚性表面形变,限制了其在标准裤装或T恤穿搭场景的应用。本文提出ReLoo方法,可从真实环境下的单目视频中重建出高质量的松垮衣物人体模型。首先建立分层神经人体表示,将穿衣人体分解为内层神经躯干与外层衣物;在此基础上引入非层级虚拟骨骼形变模块,使衣物层可自由运动,从而精确恢复其非刚性形变。通过多层可微分体渲染实现人体与衣物形状、外观及形变的全局联合优化。为评估方法性能,我们搭建多视角采集系统,记录动态形变衣物的真实场景数据。在既有数据集和新构建数据集上的实验均表明,ReLoo在室内与真实场景视频中均显著优于现有方法。
原文摘要 · Abstract (English)
While previous years have seen great progress in the 3D reconstruction of humans from monocular videos, few of the state-of-the-art methods are able to handle loose garments that exhibit large non-rigid surface deformations during articulation. This limits the application of such methods to humans that are dressed in standard pants or T-shirts. Our method, ReLoo, overcomes this limitation and reconstructs high-quality 3D models of humans dressed in loose garments from monocular in-the-wild videos. To tackle this problem, we first establish a layered neural human representation that decomposes clothed humans into a neural inner body and outer clothing. On top of the layered neural representation, we further introduce a non-hierarchical virtual bone deformation module for the clothing layer that can freely move, which allows the accurate recovery of non-rigidly deforming loose clothing. A global optimization jointly optimizes the shape, appearance, and deformations of the human body and clothing via multi-layer differentiable volume rendering. To evaluate ReLoo, we record subjects with dynamically deforming garments in a multi-view capture studio. This evaluation, both on existing and our novel dataset, demonstrates ReLoo's clear superiority over prior art on both indoor datasets and in-the-wild videos.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。