用图神经网络捕捉面部视频中的生理周期性,提升无接触心率检测精度。
Reperio-rPPG: Relational Temporal Graph Neural Networks for Periodicity Learning in Remote Physiological Measurement
- 构建关系时序图网络,显式建模生理信号的周期结构。
- 在三个数据集上达到最新最好性能,对运动和光照变化鲁棒。
- 新增CutMix增强策略,改善模型泛化能力,适合实际场景应用。
远程光电容积脉搏波描记法(rPPG)是一种新兴的非接触式生理传感技术,通过分析人脸视频中的微弱颜色变化来估计心率与呼吸频率。该方法因可扩展性和便利性,在远程医疗、情感计算、驾驶员疲劳检测及健康监测等领域广受关注。尽管进展显著,以往方法常忽视生理信号的内在周期性,限制了其在真实场景中对细微时间动态的捕捉能力。为此,本文提出Reperio-rPPG框架,将关系卷积网络与图变压器结合,有效建模生理信号的周期结构。针对现有rPPG数据集多样性不足的问题,进一步引入定制化的CutMix增强策略以提升模型泛化能力。在PURE、UBFC-rPPG和MMPD三个主流基准数据集上的大量实验表明,Reperio-rPPG不仅达到当前最优性能,且在多种运动状态(静止、旋转、说话、行走)和光照条件(自然光、低亮度LED、高亮度LED)下均表现出优异鲁棒性。代码已公开于https://github.com/deconasser/Reperio-rPPG。
原文摘要 · Abstract (English)
Remote photoplethysmography (rPPG) is an emerging contactless physiological sensing technique that leverages subtle color variations in facial videos to estimate vital signs such as heart rate and respiratory rate. This non-invasive method has gained traction across diverse domains, including telemedicine, affective computing, driver fatigue detection, and health monitoring, owing to its scalability and convenience. Despite significant progress in remote physiological signal measurement, a crucial characteristic - the intrinsic periodicity - has often been underexplored or insufficiently modeled in previous approaches, limiting their ability to capture fine-grained temporal dynamics under real-world conditions. To bridge this gap, we propose Reperio-rPPG, a novel framework that strategically integrates Relational Convolutional Networks with a Graph Transformer to effectively capture the periodic structure inherent in physiological signals. Additionally, recognizing the limited diversity of existing rPPG datasets, we further introduce a tailored CutMix augmentation to enhance the model's generalizability. Extensive experiments conducted on three widely used benchmark datasets - PURE, UBFC-rPPG, and MMPD - demonstrate that Reperio-rPPG not only achieves state-of-the-art performance but also exhibits remarkable robustness under various motion (e.g., stationary, rotation, talking, walking) and illumination conditions (e.g., nature, low LED, high LED). The code is publicly available at https://github.com/deconasser/Reperio-rPPG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。