arXiv:2601.04520cs.CV2026-01中稿 · IEEE Transactions …被引 5

用可微渲染提升人脸纹理细节,让生成图像更像真实输入。

FaceRefiner: High-Fidelity Facial Texture Refinement with Differentiable Rendering-based Style Transfer

  • 以3D纹理为风格、生成图为内容,结合可微渲染实现多层级风格迁移。
  • 在多个人脸数据集上,纹理质量和身份一致性均优于现有方法。
  • 适合需要高保真人脸重建与身份一致性的虚拟人、数字孪生场景。

近期的人脸纹理生成方法通常通过深度网络合成图像内容并填充到UV地图中,从而从单张图像生成完整纹理。然而,合成的纹理UV地图通常来自训练数据或2D生成器构建的空间,限制了对野外输入图像的泛化能力,导致生成结果在细节、结构和身份上与输入不一致。本文提出一种基于风格迁移的面部纹理精炼方法FaceRefiner。该方法将3D采样纹理视为风格,将纹理生成模型输出作为内容,期望将照片级真实感风格从风格图像迁移到内容图像。不同于现有仅传递高层和中层信息的风格迁移方法,我们的方法引入可微渲染,同步传递可见人脸区域的低层(像素级)信息。这种多层级信息迁移的优势在于能有效保留输入图像的细节、结构和语义。在Multi-PIE、CelebA和FFHQ数据集上的大量实验表明,所提精炼方法相比当前最优方法,在纹理质量与身份保持能力方面均有显著提升。

原文摘要 · Abstract (English)

Recent facial texture generation methods prefer to use deep networks to synthesize image content and then fill in the UV map, thus generating a compelling full texture from a single image. Nevertheless, the synthesized texture UV map usually comes from a space constructed by the training data or the 2D face generator, which limits the methods' generalization ability for in-the-wild input images. Consequently, their facial details, structures and identity may not be consistent with the input. In this paper, we address this issue by proposing a style transfer-based facial texture refinement method named FaceRefiner. FaceRefiner treats the 3D sampled texture as style and the output of a texture generation method as content. The photo-realistic style is then expected to be transferred from the style image to the content image. Different from current style transfer methods that only transfer high and middle level information to the result, our style transfer method integrates differentiable rendering to also transfer low level (or pixel level) information in the visible face regions. The main benefit of such multi-level information transfer is that, the details, structures and semantics in the input can thus be well preserved. The extensive experiments on Multi-PIE, CelebA and FFHQ datasets demonstrate that our refinement method can improve the texture quality and the face identity preserving ability, compared with state-of-the-arts.

人脸生成风格迁移可微渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。