用物理模拟实现可真实动的3D虚拟人,能应对新动作不崩溃。
MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics
- 用材料点法+各向异性模型模拟衣物复杂形变与身体接触。
- 在多个数据集上动态精度、渲染质量、鲁棒性均超越现有方法。
- 零样本泛化到未见交互,适合影视/游戏高保真角色生成。
尽管从视觉观测构建3D虚拟人已取得显著进展,但如何为松散衣物建模真实物理动力学仍是难题。现有基于物理模拟的方法或精度不足,或对新动画输入缺乏鲁棒性。本文提出MPMAvatar框架,从多视角视频重建3D人体,支持高保真自由视角渲染与鲁棒动画。核心思想是采用材料点法(Material Point Method)模拟器,并引入各向异性本构模型与新型碰撞处理算法,精准刻画衣物复杂变形及与身体的接触。结合可渲染的规范虚拟人结构,使用3D高斯泼溅实现带半阴影的高质量渲染。实验表明,MPMAvatar在动态建模精度、渲染准确性和鲁棒性效率上显著优于现有最先进方法。此外,首次实现零样本泛化至未见交互,突破了以往学习型方法因仿真泛化能力有限而无法实现的瓶颈。
原文摘要 · Abstract (English)
While there has been significant progress in the field of 3D avatar creation from visual observations, modeling physically plausible dynamics of humans with loose garments remains a challenging problem. Although a few existing works address this problem by leveraging physical simulation, they suffer from limited accuracy or robustness to novel animation inputs. In this work, we present MPMAvatar, a framework for creating 3D human avatars from multi-view videos that supports highly realistic, robust animation, as well as photorealistic rendering from free viewpoints. For accurate and robust dynamics modeling, our key idea is to use a Material Point Method-based simulator, which we carefully tailor to model garments with complex deformations and contact with the underlying body by incorporating an anisotropic constitutive model and a novel collision handling algorithm. We combine this dynamics modeling scheme with our canonical avatar that can be rendered using 3D Gaussian Splatting with quasi-shadowing, enabling high-fidelity rendering for physically realistic animations. In our experiments, we demonstrate that MPMAvatar significantly outperforms the existing state-of-the-art physics-based avatar in terms of (1) dynamics modeling accuracy, (2) rendering accuracy, and (3) robustness and efficiency. Additionally, we present a novel application in which our avatar generalizes to unseen interactions in a zero-shot manner-which was not achievable with previous learning-based methods due to their limited simulation generalizability. Our project page is at: https://KAISTChangmin.github.io/MPMAvatar/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。