用概率化代码本提升视频人脸修复的时序一致性，解决闪烁问题。
DicFace: Dirichlet-Constrained Variational Codebook Learning for Temporally Coherent Video Face Restoration
- 将离散代码本改为狄利克雷分布的连续变量，实现帧间特征平滑过渡
- 在盲修复、补全和着色任务上均达到当前最佳性能，有效抑制闪烁
- 适合需要高时序一致性的视频人脸修复场景，尤其适用于预训练图像模型迁移
视频人脸修复面临在恢复精细细节的同时保持时间一致性的关键挑战。本文提出一种新方法，将预训练于高质量静态肖像的向量量化变分自编码器（VQ-VAE）扩展为视频修复框架，通过变分潜在空间建模实现跨帧特征连贯性。核心创新在于将离散代码本表示重构为狄利克雷分布的连续变量，支持面部特征在帧间的概率化转移。采用时空变换器架构联合建模帧间依赖并预测潜在分布，结合拉普拉斯约束重建损失与感知（LPIPS）正则化，提升像素精度与视觉质量。在盲人脸修复、视频补全和面部着色任务上的全面评估显示，该方法达到当前最优表现。本工作建立了一种有效范式，将高保真图像先验迁移至视频修复，解决了闪烁伪影问题。源码已开源，地址为 https://github.com/fudan-generative-vision/DicFace。
原文摘要 · Abstract (English)
Video face restoration faces a critical challenge in maintaining temporal consistency while recovering fine facial details from degraded inputs. This paper presents a novel approach that extends Vector-Quantized Variational Autoencoders (VQ-VAEs), pretrained on static high-quality portraits, into a video restoration framework through variational latent space modeling. Our key innovation lies in reformulating discrete codebook representations as Dirichlet-distributed continuous variables, enabling probabilistic transitions between facial features across frames. A spatio-temporal Transformer architecture jointly models inter-frame dependencies and predicts latent distributions, while a Laplacian-constrained reconstruction loss combined with perceptual (LPIPS) regularization enhances both pixel accuracy and visual quality. Comprehensive evaluations on blind face restoration, video inpainting, and facial colorization tasks demonstrate state-of-the-art performance. This work establishes an effective paradigm for adapting intensive image priors, pretrained on high-quality images, to video restoration while addressing the critical challenge of flicker artifacts. The source code has been open-sourced and is available at https://github.com/fudan-generative-vision/DicFace.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。