让3D头像生成可自由换光，突破光照与外观纠缠难题
HeadLighter: Disentangling Illumination in Generative 3D Gaussian Heads via Lightstage Captures
- 双分支结构分离光照不变特征与物理渲染成分
- 基于灯光阵列捕获数据，实现可控光照编辑与实时渲染
- 适合需要精细光照控制的虚拟人、影视特效场景
基于3D高斯点云的3D感知头像生成模型已实现实时、逼真且视角一致的头像合成。然而，光照与固有外观深度耦合，导致无法实现可控重光照。现有解耦方法依赖强假设进行弱监督学习，难以处理复杂光照。为此，我们提出HeadLighter，一种新型监督框架，可在头像生成模型中学习物理合理的外观与光照分解。我们设计双分支架构，分别建模光照无关的头部属性与物理驱动的渲染组件，并采用渐进式解耦训练，利用灯光阵列在受控光照下采集的多视角图像进行监督。此外，引入知识蒸馏策略生成高质量法线以提升渲染真实感。实验表明，该方法在保持高质量生成与实时渲染的同时，支持显式的光照与视角编辑。代码与数据集将公开。
原文摘要 · Abstract (English)
Recent 3D-aware head generative models based on 3D Gaussian Splatting achieve real-time, photorealistic and view-consistent head synthesis. However, a fundamental limitation persists: the deep entanglement of illumination and intrinsic appearance prevents controllable relighting. Existing disentanglement methods rely on strong assumptions to enable weakly supervised learning, which restricts their capacity for complex illumination. To address this challenge, we introduce HeadLighter, a novel supervised framework that learns a physically plausible decomposition of appearance and illumination in head generative models. Specifically, we design a dual-branch architecture that separately models lighting-invariant head attributes and physically grounded rendering components. A progressive disentanglement training is employed to gradually inject head appearance priors into the generative architecture, supervised by multi-view images captured under controlled light conditions with a light stage setup. We further introduce a distillation strategy to generate high-quality normals for realistic rendering. Experiments demonstrate that our method preserves high-quality generation and real-time rendering, while simultaneously supporting explicit lighting and viewpoint editing. We will publicly release our code and dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。