arXiv:2412.07203cs.CV2024-12被引 1

针对人脸着色不自然问题,分组件学习颜色表示并提升效果。

Learning Spatially Decoupled Color Representations for Facial Image Colorization

  • 按唇、皮肤、眼、发等面部组件分块学习颜色特征。
  • 在单/多参考图场景下均优于现有方法,无参考时也有效。
  • 适合需要精准人脸着色的应用,如影视修复与虚拟形象生成。

图像着色方法在自然图像上表现优异,但面对人脸时因人对人脸更敏感,现有方法常产生不自然、不均匀的着色结果。本文研究人脸着色任务,发现其问题源于对面部组件理解不足。为此,引入面部组件先验,提出新框架FCNet:基于人脸分割图,为每个面部组件(如唇、皮肤、眼、发)学习解耦的颜色表示。设计了色彩与空间增强策略,仅需灰度与彩色人脸图像对即可训练。训练后,FCNet可自然应用于单或多个参考图像的着色。为进一步拓展至无参考场景,额外训练两个模块:一个从灰度输入预测颜色表示,另一个从随机种子生成。大量实验表明,本方法在无参考、单参考、多参考等场景中均优于现有方法。源代码与预训练模型将公开。

原文摘要 · Abstract (English)

Image colorization methods have shown prominent performance on natural images. However, since humans are more sensitive to faces, existing methods are insufficient to meet the demands when applied to facial images, typically showing unnatural and uneven colorization results. In this paper, we investigate the facial image colorization task and find that the problems with facial images can be attributed to an insufficient understanding of facial components. As a remedy, by introducing facial component priors, we present a novel facial image colorization framework dubbed FCNet. Specifically, we learn a decoupled color representation for each face component (e.g., lips, skin, eyes, and hair) under the guidance of face parsing maps. A chromatic and spatial augmentation strategy is presented to facilitate the learning procedure, which requires only grayscale and color facial image pairs. After training, the presented FCNet can be naturally applied to facial image colorization with single or multiple reference images. To expand the application paradigms to scenarios with no reference images, we further train two alternative modules, which predict the color representations from the grayscale input or a random seed, respectively. Extensive experiments show that our method can perform favorably against existing methods in various application scenarios (i.e., no-, single-, and multi-reference facial image colorization). The source code and pre-trained models will be publicly available.

人脸着色解耦表示图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。