用文字描述生成风格化3D人脸模型,支持精确控制表情与形状。
StyleMM: Stylized 3D Morphable Face Model via Text-Driven Aligned Image Translation
- 基于文本引导的图像翻译生成风格化人脸图,作为3D模型训练目标。
- 在保持身份、表情一致的前提下,实现跨参数空间的风格迁移。
- 适合需要可控风格人脸生成的应用,如动画、虚拟角色设计。
我们提出StyleMM,一种可根据用户指定文本描述构建风格化3D可变形人脸模型(3DMM)的新框架。在预训练的网格变形网络和真实人脸纹理生成器基础上,利用扩散模型驱动的文本引导图像到图像(i2i)翻译生成风格化人脸图像,并以此作为网格渲染的风格化目标。为防止风格转换过程中身份、面部对齐或表情发生意外变化,我们引入一种显式保留源图像面部属性的风格化方法。通过保持关键属性,该方法确保在3DMM参数空间中实现一致的风格迁移。训练完成后,StyleMM可实现前向生成风格化人脸网格,且能明确控制形状、表情和纹理参数,生成具有统一顶点连接性与可动画性的网格。定量与定性评估表明,本方法在身份级人脸多样性与风格化能力方面优于现有最优方法。代码与视频见[kwanyun.github.io/stylemm_page]。
原文摘要 · Abstract (English)
We introduce StyleMM, a novel framework that can construct a stylized 3D Morphable Model (3DMM) based on user-defined text descriptions specifying a target style. Building upon a pre-trained mesh deformation network and a texture generator for original 3DMM-based realistic human faces, our approach fine-tunes these models using stylized facial images generated via text-guided image-to-image (i2i) translation with a diffusion model, which serve as stylization targets for the rendered mesh. To prevent undesired changes in identity, facial alignment, or expressions during i2i translation, we introduce a stylization method that explicitly preserves the facial attributes of the source image. By maintaining these critical attributes during image stylization, the proposed approach ensures consistent 3D style transfer across the 3DMM parameter space through image-based training. Once trained, StyleMM enables feed-forward generation of stylized face meshes with explicit control over shape, expression, and texture parameters, producing meshes with consistent vertex connectivity and animatability. Quantitative and qualitative evaluations demonstrate that our approach outperforms state-of-the-art methods in terms of identity-level facial diversity and stylization capability. The code and videos are available at [kwanyun.github.io/stylemm_page](kwanyun.github.io/stylemm_page).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。