通过查询纯净代码本,提升人脸视频中生理信号的抗干扰能力。
CodePhys: Robust Video-based Remote Physiological Measurement through Latent Codebook Querying
- 将rPPG测量转为在纯净代码本中的查询任务
- 在4个数据集上均超越现有方法,跨数据集表现更优
- 适合需要高鲁棒性远程生理监测的应用场景
远程光电容积脉搏波描记(rPPG)旨在从人脸视频中非接触式提取生理信号,具有广泛应用潜力。现有方法多通过神经网络直接提取视频特征进行心率估计,但在真实场景下易受相机噪声、模糊和运动伪影等非生理因素干扰,导致信号失真。本文提出CodePhys方法,将rPPG测量视为在由真实PPG信号构建的无噪代理空间(即代码本)中进行特征查询的任务。将噪声干扰的rPPG特征作为查询,通过匹配代码本中的纯净特征生成高质量信号。方法还引入空间感知编码器与空间注意力机制,突出生理活跃区域,并采用知识蒸馏损失降低非周期性视觉干扰影响。在四个基准数据集上的实验表明,CodePhys在同数据集与跨数据集设置下均优于现有最先进方法。
原文摘要 · Abstract (English)
Remote photoplethysmography (rPPG) aims to measure non-contact physiological signals from facial videos, which has shown great potential in many applications. Most existing methods directly extract video-based rPPG features by designing neural networks for heart rate estimation. Although they can achieve acceptable results, the recovery of rPPG signal faces intractable challenges when interference from real-world scenarios takes place on facial video. Specifically, facial videos are inevitably affected by non-physiological factors (e.g., camera device noise, defocus, and motion blur), leading to the distortion of extracted rPPG signals. Recent rPPG extraction methods are easily affected by interference and degradation, resulting in noisy rPPG signals. In this paper, we propose a novel method named CodePhys, which innovatively treats rPPG measurement as a code query task in a noise-free proxy space (i.e., codebook) constructed by ground-truth PPG signals. We consider noisy rPPG features as queries and generate high-fidelity rPPG features by matching them with noise-free PPG features from the codebook. Our approach also incorporates a spatial-aware encoder network with a spatial attention mechanism to highlight physiologically active areas and uses a distillation loss to reduce the influence of non-periodic visual interference. Experimental results on four benchmark datasets demonstrate that CodePhys outperforms state-of-the-art methods in both intra-dataset and cross-dataset settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。