仅用一张照片2分钟生成可动画3D角色，支持实时对话
Make-A-Character 2: Animatable 3D Character Generation From a Single Image
- 通过光照修正与肤色调优，提升输入图到3D的还原精度
- 采用分层网络捕捉面部细节，自适应骨骼校准实现自然表情
- 融合Transformer生成同步语音的动作，适合游戏与数字人应用
本文介绍Make-A-Character 2，一个从单张人像照片生成高质量3D角色的先进系统，适用于游戏开发与数字人场景。该系统在前代基础上引入多项改进：利用IC-Light方法校正输入图像中的非理想光照，并通过神经网络进行肤色调优，使照片与游戏引擎渲染结果的皮肤色调一致；采用分层表示网络捕捉高频率面部结构，并实施自适应骨骼校准，实现精准且富有表现力的面部动画；整个图像到3D角色生成过程耗时不足2分钟。此外，系统还基于Transformer架构生成伴随语音的面部表情与手势动作，支持与生成角色的实时对话。上述技术已集成至我们的对话式AI虚拟人产品中。
原文摘要 · Abstract (English)
This report introduces Make-A-Character 2, an advanced system for generating high-quality 3D characters from single portrait photographs, ideal for game development and digital human applications. Make-A-Character 2 builds upon its predecessor by incorporating several significant improvements for image-based head generation. We utilize the IC-Light method to correct non-ideal illumination in input photos and apply neural network-based color correction to harmonize skin tones between the photos and game engine renders. We also employ the Hierarchical Representation Network to capture high-frequency facial structures and conduct adaptive skeleton calibration for accurate and expressive facial animations. The entire image-to-3D-character generation process takes less than 2 minutes. Furthermore, we leverage transformer architecture to generate co-speech facial and gesture actions, enabling real-time conversation with the generated character. These technologies have been integrated into our conversational AI avatar products.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。