arXiv:2506.21855cs.CV2025-06被引 1

用自监督方法从人脸视频中学习脉搏信号特征,提升无接触测心率精度。

Periodic-MAE: Periodic Video Masked Autoencoder for rPPG Estimation

  • 设计周期性掩码策略,让模型聚焦脉搏的准周期性变化。
  • 在四个基准数据集上均优于现有方法,跨数据集表现提升显著。
  • 适合做无接触生理监测、医疗健康应用的研究者参考。

本文提出Periodic-MAE,一种自监督框架,用于从未标注的人脸视频中学习可泛化的周期性生理信号时空表示。该方法基于掩码自编码器(MAE),通过重建被遮蔽的视频帧块来学习高维面部表征,无需依赖远程光体积描记(rPPG)的特定标注。为显式对齐表征学习与rPPG特性,我们提出基于视频重采样的周期性感知帧掩码策略,使编码器能捕捉与脉搏估计相关的准周期性时间模式。同时,将生理频带限制引入MAE预训练框架,利用脉搏信号在频域中的稀疏性,引导学习到的表征趋向生理意义明确的模式。预训练后,将学习到的表征迁移至下游rPPG估计任务,其中编码器作为通用特征提取器,从人脸视频中恢复脉搏相关信号。我们在PURE、UBFC-rPPG、MMPD和V4V四个基准数据集上进行了广泛实验,并在真实世界中采集的非受限光照与运动条件下的rPPG数据集上评估。结果表明,Periodic-MAE在挑战性的跨数据集及真实场景设置下持续提升rPPG估计性能。代码已开源:https://github.com/ziiho08/Periodic-MAE。

原文摘要 · Abstract (English)

In this paper, we propose Periodic-MAE, a self-supervised framework for learning generalizable spatio-temporal representations of periodic physiological signals from unlabeled facial videos. The proposed method leverages a masked autoencoder (MAE), which learns high-dimensional facial representations by reconstructing masked video tokens without relying on remote photoplethysmography (rPPG) specific supervision. To explicitly align representation learning with the characteristics of rPPG, we introduce a periodicity-aware frame masking strategy based on video resampling, enabling the encoder to learn representations that capture quasi-periodic temporal patterns relevant to pulse signal estimation. In addition, physiological bandlimit constraints are integrated into the MAE pre-training framework, exploiting the sparsity of pulse signals in the frequency domain to guide the learned representations toward physiologically meaningful patterns. After pre-training, the learned representations are transferred to downstream rPPG estimation, where the encoder serves as a generic feature extractor for recovering pulse-related signals from facial videos. We conduct extensive experiments on four benchmark datasets, including PURE, UBFC-rPPG, MMPD, and V4V. Moreover, we evaluate the proposed approach on a real-world rPPG dataset collected under unconstrained lighting conditions and subject motion. Experimental results demonstrate that Periodic-MAE consistently improves rPPG estimation performance, particularly in challenging cross-dataset and real-world evaluation settings. Our code is available at https://github.com/ziiho08/Periodic-MAE.

rPPG自监督视频分析生理信号

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。