提出首个轻量级端到端rPPG模型,可在极端光照下准确提取生理信号。
Remote Photoplethysmography in Real-World and Extreme Lighting Scenarios
- 采用视频变压器架构,融合全局干扰共享与自监督解耦机制。
- 在真实户外场景中实现高精度心率预测,跨数据集性能领先。
- 适合野外监测、可穿戴设备部署等实际应用需求。
生理活动可通过面部图像的细微变化体现。尽管这些变化肉眼难以察觉,但计算机视觉方法可捕捉,由此衍生的远程光电容积脉搏波描记法(rPPG)展现出巨大潜力。然而,现有研究多依赖空间皮肤识别和时间节律交互,在理想光照条件下表现良好,但在真实复杂环境及极端光照下性能显著下降。本文提出一种端到端视频变压器模型,旨在消除外部时变干扰(无论其是否掩盖微弱生物信号或作为周期性扰动影响训练)。具体实现中,通过全局干扰共享、主体背景参考和自监督解耦来抑制干扰,并结合时空滤波、重建引导、频域与生物先验约束进行有效学习。据我们所知,这是首个基于自然人脸视频、面向真实室外场景的鲁棒rPPG模型,且轻量化便于部署。大量实验表明,该模型在多个数据集和场景中均表现出色,具备竞争力。
原文摘要 · Abstract (English)
Physiological activities can be manifested by the sensitive changes in facial imaging. While they are barely observable to our eyes, computer vision manners can, and the derived remote photoplethysmography (rPPG) has shown considerable promise. However, existing studies mainly rely on spatial skin recognition and temporal rhythmic interactions, so they focus on identifying explicit features under ideal light conditions, but perform poorly in-the-wild with intricate obstacles and extreme illumination exposure. In this paper, we propose an end-to-end video transformer model for rPPG. It strives to eliminate complex and unknown external time-varying interferences, whether they are sufficient to occupy subtle biosignal amplitudes or exist as periodic perturbations that hinder network training. In the specific implementation, we utilize global interference sharing, subject background reference, and self-supervised disentanglement to eliminate interference, and further guide learning based on spatiotemporal filtering, reconstruction guidance, and frequency domain and biological prior constraints to achieve effective rPPG. To the best of our knowledge, this is the first robust rPPG model for real outdoor scenarios based on natural face videos, and is lightweight to deploy. Extensive experiments show the competitiveness and performance of our model in rPPG prediction across datasets and scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。