用笔记本摄像头生成逼真人像，无需专业设备和云端算力
High-Fidelity Human Avatars from Laptop Webcams using Edge Compute
- 结合3D人脸模型与可微渲染，从低分辨率摄像头捕捉数据
- 在移动端AMD处理器上实现高质量纹理生成与动画绑定
- 适合视频会议等实时场景，无需外接设备或云端支持
逼真人像在诸多领域有广泛应用，但传统方法依赖昂贵的专业摄像设备和大量人工投入。近期研究实现了通过智能手机的RGB与红外传感器自动构建人像，但仍需高分辨率摄像头及高性能服务器进行处理。现代视频会议等应用亟需在消费级笔记本摄像头和有限本地算力下生成高保真人像。本文提出一种新方法，基于3D可变形模型、关键点检测、摄影真实感纹理生成对抗网络（GAN）及可微渲染，克服低质量摄像头输入与边缘计算限制。我们构建了一个全自动系统，在AMD移动处理器上实现高保真可动画人像生成，满足实际部署需求。
原文摘要 · Abstract (English)
Photo-realistic human avatars have broad applications, yet high-fidelity avatar generation has traditionally required expensive professional camera rigs and extensive artistic labor. Recent research has enabled constructing them automatically from smartphones with RGB and IR sensors, however, these new methods still rely on high-resolution cameras on modern smartphones and often require offloading the processing to powerful servers with GPUs. Modern applications such as video conferencing call for the ability to generate these avatars from consumer-grade laptop webcams using limited compute available on-device. In this work, we develop a novel method based on 3D morphable models, landmark detection, photorealistic texture GANs, and differentiable rendering to tackle the problem of low webcam image quality and edge computation. We build an automatic system to generate high-fidelity animatable avatars under these limitations, leveraging the compute capabilities of AMD mobile processors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。