仅用10帧人脸视频，快速提升低带宽视频通话画质
FSFVE: Few Shot Compressed Face Video Enhancement
- 基于少量人脸帧训练轻量模型，实现快速增强
- 100秒内完成训练，显著提升压缩视频的清晰度与观感
- 无需修改现有应用，可在普通笔记本上实时运行
视频通话已成为全球主流通信方式，但受带宽限制，仍有许多人面临通话质量差的问题，尽管设备齐全。本文提出一种新型框架，仅需10帧人脸视频即可在100秒内快速训练出增强模型，适用于通话前或通话中。该模型可作为独立层部署于现有视频会议软件（如Zoom）之上，无需修改原程序，在图像显示前实时拦截并优化画质。模型设计为能在典型笔记本电脑的CPU上实时运行。实验表明，该方法在量化指标和主观感知上均显著提升了压缩人脸视频的质量。代码已开源：https://github.com/varun-jois/FSFVE。
原文摘要 · Abstract (English)
Videocalling has become a popular form of communication in the world today, with many companies providing free services for it. However, there are still millions of people around the world that experience poor quality videocalls due to limitations in bandwidth. This despite, most people having the required hardware. In this paper we present a novel framework for enhancing highly compressed videocalls. We show, that with as little as 10 frames of the face, we can rapidly (in under 100 seconds) train a model to enhance that instance of the videocall. The model can be trained either prior to or during the call, enhancing the rest of the call by producing better quality video. The video conferencing application need not be modified - it can be off the shelf with our system as a layer on top that trains quickly then simply lets the video conferencing application (e.g. Zoom) run as usual, where our system intercepts and improves images before they are displayed. The model is designed to run in realtime on low-compute devices such as a typical laptop CPU. Experimentally, we show that the model significantly improves quality of compressed face video both quantitatively as well as perceptually. Code can be found at https://github.com/varun-jois/FSFVE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。