实时人脸光照重制，兼顾真实感与身份一致性。
FusionRelight: Relighting Portraits in Real Time via Hybrid Domain Knowledge Fusion
- 融合物理、反照率与真实感先验，分阶段训练优化
- 在OLAT基准上各项指标领先，实测延迟仅1.82毫秒
- 适合影视特效、直播美颜等需要快速高质量光照调整的场景
人脸光照重制是低层视觉问题，需同时考虑物理合理光照迁移、身份保持与紧凑实时推理。迭代扩散方法虽能生成精细细节，但随机推理成本高，难以用于确定性视频流；基于物理的重制方法虽保留身份，但受控合成或光场监督难迁移到非约束相机。本文提出混合领域知识融合（HDKF）框架，从合成数据、单光源逐次（OLAT）及真实场景数据中学习互补的物理、反照率与真实感先验，并将源路径监督蒸馏至轻量学生模型，输入为退化图像，标签为清晰教师输出。训练采用像素对齐的RGB、反照率与法线监督，构建物理基础的低层重制模拟环境。在预留的OLAT基准测试中，HDKF在MSE、PSNR、SSIM上均优于对比方法，且在LPIPS上仍具竞争力。蒸馏模型在512x512分辨率下实现实时运行，RTX 2060上延迟11.89毫秒，RTX 4090上低至1.82毫秒。
原文摘要 · Abstract (English)
Portrait relighting is a low-level vision problem in which physically plausible illumination transfer, identity preservation, and compact real-time inference must be considered together. Iterative diffusion-style methods can synthesize fine detail, but stochastic inference and cost complicate deterministic live video creation; physically grounded relighting preserves identity, but controlled synthetic or light-stage supervision transfers poorly to unconstrained cameras. We present Hybrid Domain Knowledge Fusion (HDKF), a relighting-specific training framework that learns complementary physics, reflectance, and realism priors from synthetic, One-Light-at-A-Time (OLAT), and in-the-wild data, then distills their source-routed supervision into a compact student with clean teacher labels and degraded student inputs. The framework is trained with pixel-aligned RGB, albedo, and normal supervision, providing a simulation substrate for physically grounded low-level relighting. On a held-out OLAT benchmark, HDKF obtains the best MSE, PSNR, and SSIM among evaluated methods while remaining competitive in LPIPS. The distilled model runs in real time at 512x512, reaching 11.89 ms on an RTX 2060 and 1.82 ms on an RTX 4090.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。