用神经网络统一控制3D虚拟人，键盘操作即可生成动态形象。
UNICA: A Unified Neural Framework for Controllable 3D Avatars
- 不依赖骨骼，通过2D位置图和扩散模型生成动作帧
- 支持长时序自回归生成，自然模拟头发衣物动态
- 适合游戏、元宇宙等需要快速生成虚拟人的场景
可控3D虚拟人广泛应用于3D游戏、元宇宙和AR/VR场景。传统构建流程复杂，需经历外观建模、运动规划、绑定和物理仿真等多个环节。本文提出UNICA（UNIfied neural Controllable Avatar），一种无需骨架的生成式框架，将所有控制模块统一为单一神经网络。给定类似视频游戏的键盘输入，UNICA通过作用于2D位置图的动作条件扩散模型生成下一帧3D虚拟人的几何结构；随后利用点变换器将结果映射至3D高斯溅射，实现高保真自由视角渲染。该方法无需手动设计物理仿真，天然捕捉发丝与松散衣物的动态行为，并支持超长时序自回归生成。据我们所知，UNICA是首个统一‘运动规划、绑定、物理仿真与渲染’全流程的模型。代码已开源：https://github.com/zjh21/UNICA。
原文摘要 · Abstract (English)
Controllable 3D human avatars have found widespread applications in 3D games, the metaverse, and AR/VR scenarios. The conventional approach to creating such a 3D avatar requires a lengthy, intricate pipeline encompassing appearance modeling, motion planning, rigging, and physical simulation. In this paper, we introduce UNICA (UNIfied neural Controllable Avatar), a skeleton-free generative model that unifies all avatar control components into a single neural framework. Given keyboard inputs akin to video game controls, UNICA generates the next frame of a 3D avatar's geometry through an action-conditioned diffusion model operating on 2D position maps. A point transformer then maps the resulting geometry to 3D Gaussian Splatting for high-fidelity free-view rendering. Our approach naturally captures hair and loose clothing dynamics without manually designed physical simulation, and supports extra-long autoregressive generation. To the best of our knowledge, UNICA is the first model to unify the workflow of "motion planning, rigging, physical simulation, and rendering". Code is released at https://github.com/zjh21/UNICA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。