提出可实时部署的无监督人脸修复模型,兼顾画质与推理效率。
CFRNet: Cycle-Consistent Fixed-Point Training for Real-Time Blind Face Restoration on Consumer Embedded NPUs

- 采用循环一致定点训练,使模型迭代修复后结果不变。
- 在300张测试图上LPIPS达0.250(三轮),比单轮降低31%。
- 支持硬件直推,可在车机端实现实时修复,无需重新训练。
消费级设备上的人脸盲修复需在画质、速度和内存间权衡。强方法如GFPGAN和CodeFormer虽感知质量高,但依赖大型生成先验及注意力、代码本查找等难以在小型神经处理单元(NPU)上编译量化。小卷积网络虽快,却易过平滑并留下眼鼻口区域伪影。本文提出CFRNet,一种2.0M参数的ResNet风格修复器,适用于256×256分辨率的嵌入式设备。核心为循环一致定点训练(CCFP),通过多轮循环监督、幂等性损失和再退化循环损失,训练模型成为固定点算子,重复应用不改变输出。所有基线均在相同256×256分辨率下从头重训。在300图像测试集上,CFRNet实现最优感知分数(三轮时LPIPS为0.250,较一轮降低31%),且两轮时PSNR与SSIM最优。在海思Hi3402 NPU上以INT8运行,每轮约23ms,而同类模型无法编译至该芯片。循环次数k作为无需重训练的质量调节旋钮:PSNR最佳于k=2,LPIPS持续提升至k=3。该方法亦适用于更易部署的普通CNN,在车载驾驶员监测板上实现实时运行。
原文摘要 · Abstract (English)
Blind face restoration on consumer devices has to balance image quality against speed and memory. Strong methods such as GFPGAN and CodeFormer give good perceptual quality, but they rely on large pretrained generative priors and on operators such as attention, codebook lookup, and style modulation that are hard to compile and quantize on the small neural processing units (NPUs) used in consumer hardware. Small convolutional restorers run fast enough, but they tend to over-smooth and to leave artifacts around the eyes, nose, and mouth. We present CFRNet, a 2.0,M-parameter ResNet-style restorer for on-device use at $256\times256$, the common face-crop size on consumer NPUs. The main idea is Cycle-Consistent Fixed-Point Training (CCFP). Instead of training the network for one pass and then running it several times by hand, we train it to act as a fixed-point operator, so that applying it again to a restored face does not change the face. CCFP uses three training losses, namely progressive multi-cycle supervision, an idempotence loss, and a re-degradation cycle loss, and it adds no cost at inference. To compare fairly under our deployment limits, we retrain all baselines from scratch at the same $256\times256$ resolution. On a 300-image test set, CFRNet reaches the best perceptual score (LPIPS 0.250 at three cycles, which is 31% lower than one cycle) and also the best PSNR and SSIM at two cycles. It runs in about 23,ms per cycle in INT8 on a HiSilicon Hi3402 NPU, while the same baselines cannot be compiled to that chip. The cycle count $k$ acts as a simple quality knob that needs no retraining: PSNR is best at $k\!=\!2$ and LPIPS keeps improving up to $k\!=\!3$. We further show that the same idea works with a plain CNN that is even easier to deploy, and we run the model in real time on an in-car driver-monitoring board.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。