利用多视角人脸信息协同恢复高清图像并提升识别精度
Collaborative Feature Aggregation for Face Super-Resolution and Robust Re-Identification
- 通过变压器模型融合多时序/多视角人脸特征,统一身份表征
- 在低质量图像下实现更清晰的面部重建,重识别准确率显著提升
- 适合跨视角、低分辨率场景下的人脸识别与重建任务
我们提出一种新颖的协同方法,用于从序列或多视角人脸图像中进行面部超分辨率(SR)和鲁棒的人体再识别。传统超分辨率方法在处理低质量图像时常出现模糊和失真。基于图像或视频的面部超分辨率方法若依赖面部关键点或分割也面临类似挑战。为克服这些局限,我们引入一种基于变压器的协同特征聚合方法,利用时间或视角间相关的人脸观测,统一多序列或多视角数据中的身份特征,使同一人多个序列的人脸共同参与高精度共性面部特征估计。此外,我们设计了一种级联超分辨率网络,逐步恢复目标面部的高分辨率图像,并实现面部特征的渐进式统一。统一的身份表征进一步应用于人体再识别场景,即使在严重图像退化下也能实现精准匹配。大量实验结果表明,该方法优于现有最先进方法,在面部超分辨率和再识别性能上均实现稳定提升。本工作展示了从多个人脸输入中联合重建身份与渐进式图像修复对下游视觉识别任务的有效性。
原文摘要 · Abstract (English)
We propose a novel collaborative approach for face super-resolution (SR) and robust person re-identification from sequential or multi-view facial images. Traditional SR methods often suffer from blurring and distortion in faces recovered from poor-quality images due to low resolution. Image- and video-based facial SR methods using facial landmarks or segmentation also have similar challenges. To overcome these limitations, we leverage multiple correlated facial observations, across time or viewpoints, by introducing a transformer-based collaborative feature aggregation method that unifies identity features from multi-sequence or multi-view data. This allows faces in multiple sequences of an individual to contribute to accurately estimating common facial features. Furthermore, we propose a cascade SR network to progressively restore the high-resolution image of the target's face with gradual facial feature unification. The unified identity representation is further utilized in person re-identification scenarios, enabling accurate matching even under severe image degradation. The exhaustive experimental results and comparisons show that our method outperforms other state-of-the-art methods, demonstrating consistent improvements in both face super-resolution and re-identification performance. Our work highlights the effectiveness of joint identity reconstruction and progressive image restoration from multiple facial inputs in enhancing downstream visual recognition tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。