用手机加偏振片就能拍出可随意打光的高精度3D人脸视频，成本极低。
LightHeadEd: Relightable & Editable Head Avatars from a Smartphone
- 用单摄像头+偏振滤镜同步采集偏振视频，分离皮肤漫反射与高光成分。
- 结合2D高斯和参数化头模型，实现实时渲染且保留精细几何细节。
- 能分离光照、表情、姿态，适合虚拟人、元宇宙等场景应用。
传统生成逼真可动画化、可重打光的3D人脸形象需依赖昂贵的Lightstage系统与多校准相机，难以普及。为此，我们提出一种新颖、低成本的方法，仅使用带偏振滤镜的智能手机即可创建高质量可重打光的人脸三维模型。在暗室中，通过单点光源同时捕获交叉偏振与平行偏振视频流，动态捕捉面部表演时皮肤的漫反射与镜面反射成分。我们引入一种混合表示：将2D高斯嵌入参数化头模型的UV空间，实现高效实时渲染并保留高保真几何细节。基于学习的神经分析-合成管道将姿态与表情相关的几何偏移与外观解耦，分解出反照率、法线、镜面度的UV纹理图及环境图。我们构建了一个包含多种受试者进行多样化表情与头部运动的独特数据集。
原文摘要 · Abstract (English)
Creating photorealistic, animatable, and relightable 3D head avatars traditionally requires expensive Lightstage with multiple calibrated cameras, making it inaccessible for widespread adoption. To bridge this gap, we present a novel, cost-effective approach for creating high-quality relightable head avatars using only a smartphone equipped with polaroid filters. Our approach involves simultaneously capturing cross-polarized and parallel-polarized video streams in a dark room with a single point-light source, separating the skin's diffuse and specular components during dynamic facial performances. We introduce a hybrid representation that embeds 2D Gaussians in the UV space of a parametric head model, facilitating efficient real-time rendering while preserving high-fidelity geometric details. Our learning-based neural analysis-by-synthesis pipeline decouples pose and expression-dependent geometrical offsets from appearance, decomposing the surface into albedo, normal, and specular UV texture maps, along with the environment maps. We collect a unique dataset of various subjects performing diverse facial expressions and head movements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。