用轻量结构提升远程心率检测精度,无需复杂Transformer
TYrPPG: Uncomplicated and Enhanced Learning Capability rPPG for Remote Heart Rate Estimation
- 基于Mambaout设计新型门控视频块,融合2D/3D-CNN高效分析视频
- 在MAHNOB-HCI和UBFC数据集上均达领先性能,误差低于1.5bpm
- 适合资源受限场景的实时心率监测,代码开源可复现
远程光电容积脉搏波描记法(rPPG)可通过RGB视频非侵入式提取生理信号,在心率检测中具有低成本、无创等优势。现有rPPG模型多依赖Transformer模块,计算效率较低。近期Mamba模型在自然语言处理中表现出高效性,但其核心状态空间模块(SSM)在视觉任务中被证明非必要。为此,本文提出一种新型rPPG算法TYrPPG,采用Mambaout结构为基础,设计新颖的门控视频理解块(GVB),融合2D-CNN与3D-CNN以增强视频表征能力。同时引入综合监督损失函数(CSL)及其弱监督变体,显著提升模型学习能力。实验表明,TYrPPG在MAHNOB-HCI与UBFC等常用数据集上达到当前最优性能,平均误差低于1.5bpm,验证了其在远程心率估计中的潜力与优越性。源码已公开于https://github.com/Taixi-CHEN/TYrPPG。
原文摘要 · Abstract (English)
Remote photoplethysmography (rPPG) can remotely extract physiological signals from RGB video, which has many advantages in detecting heart rate, such as low cost and no invasion to patients. The existing rPPG model is usually based on the transformer module, which has low computation efficiency. Recently, the Mamba model has garnered increasing attention due to its efficient performance in natural language processing tasks, demonstrating potential as a substitute for transformer-based algorithms. However, the Mambaout model and its variants prove that the SSM module, which is the core component of the Mamba model, is unnecessary for the vision task. Therefore, we hope to prove the feasibility of using the Mambaout-based module to remotely learn the heart rate. Specifically, we propose a novel rPPG algorithm called uncomplicated and enhanced learning capability rPPG (TYrPPG). This paper introduces an innovative gated video understanding block (GVB) designed for efficient analysis of RGB videos. Based on the Mambaout structure, this block integrates 2D-CNN and 3D-CNN to enhance video understanding for analysis. In addition, we propose a comprehensive supervised loss function (CSL) to improve the model's learning capability, along with its weakly supervised variants. The experiments show that our TYrPPG can achieve state-of-the-art performance in commonly used datasets, indicating its prospects and superiority in remote heart rate estimation. The source code is available at https://github.com/Taixi-CHEN/TYrPPG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。