将人脸分解为独立组件,实现精准无副作用的编辑。
COSY: Compositional 3DGS Synthesis for Disentangled Human Head Editing

- 按区域独立生成头发、皮肤等部件,互不干扰。
- 仅需颜色信息即可编辑,无需分割图或先验形状。
- 适合需要精细控制人脸属性的研究与应用。
近期基于3D高斯溅射(3DGS)的生成对抗网络可实时合成逼真的人脸3D模型,且在身份与外观上表现多样。然而,控制如发色、眼镜等特定语义属性仍具挑战,因在纠缠的隐空间中编辑常引发身份或外观的意外变化。尽管已有方法尝试通过估计仅改变特定特征的方向来解耦隐空间,但无法保证完全解耦,且通常依赖预训练分类器。本文提出一种新生成器架构,能完全独立地合成头发、皮肤、眼镜和躯干等组件,使修改某一区域的隐向量时其余部分保持不变。此外,我们仅需稀疏信息(如发色或肤色)即可实现编辑,无需分割掩码或几何先验,常见于以往工作。为确保编辑时形状与光照一致,各独立生成器间仅通过少量上下文令牌共享必要信息,甚至可控制形状与光照而无需标注。相比现有基于GAN的生成与编辑方法,本方法在解耦性、编辑精度方面更优,视觉质量具有竞争力。
原文摘要 · Abstract (English)
Recent 3D Gaussian Splatting (3DGS) GANs for human heads synthesize and render photorealistic 3D models in real-time and offer a vast variety in identity and appearance. However, controlling specific semantic attributes such as hair color or glasses remains challenging, as edits in the entangled latent space often induce unintended changes in identity or appearance. Although there are several methods that aim to disentangle the latent space post training by estimating directions that only modify certain features, these methods cannot guarantee complete disentanglement and often require pre-trained classifiers. In our approach, we propose a new generator architecture that synthesizes components, such as hair, skin, glasses, and torso, completely independently. This allows for changing the latent vector for one region while keeping the remaining parts fixed. Further, we achieve this separation using only sparse information such as the hair or skin color, eliminating the requirement of segmentation masks or geometric priors, often seen in prior work. To ensure matching shape and lighting conditions during editing, we allow minimal shared information via context tokens between the independent generators. These tokens even allow us to control the shape and light, without any prior annotation. Compared to existing works on GAN-based generation and editing, our method shows better disentanglement, more precise editing control, and competitive visual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。