arXiv:2603.19752cs.CV2026-03被引 3

融合视频与时空图,提升无接触心率测量精度

PhysNeXt: Next-Generation Dual-Branch Structured Attention Fusion Network for Remote Photoplethysmography Measurement

  • 双分支结构同时处理原始视频和时空图特征
  • 在复杂光照与运动下仍保持稳定心率信号
  • 适合医疗健康监测与智能穿戴设备应用

远程光电容积脉搏波描记(rPPG)通过分析面部皮肤因心跳引起的微弱颜色变化,实现无接触心率等生命体征测量。现有方法主要基于原始视频的端到端建模或中间的时空图(STMap)表示。前者保留完整时空信息,能捕捉细微心跳信号,但易受运动伪影和光照变化干扰;后者将多个面部感兴趣区域的时间颜色变化压缩为二维表示,大幅降低数据量与计算复杂度,但可能丢失高频细节。为此,本文提出PhysNeXt,一种双输入深度学习框架,联合利用视频帧与STMap表示。通过引入时空差分建模单元、跨模态交互模块及结构化注意力解码器,协同增强脉搏信号提取鲁棒性。实验表明,PhysNeXt在复杂条件下实现了更稳定、更精细的rPPG信号恢复,验证了视频与STMap联合建模的有效性。代码将公开。

原文摘要 · Abstract (English)

Remote photoplethysmography (rPPG) enables contactless measurement of heart rate and other vital signs by analyzing subtle color variations in facial skin induced by cardiac pulsation. Current rPPG methods are mainly based on either end-to-end modeling from raw videos or intermediate spatial-temporal map (STMap) representations. The former preserves complete spatiotemporal information and can capture subtle heartbeat-related signals, but it also introduces substantial noise from motion artifacts and illumination variations. The latter stacks the temporal color changes of multiple facial regions of interest into compact two-dimensional representations, significantly reducing data volume and computational complexity, although some high-frequency details may be lost. To effectively integrate the mutual strengths, we propose PhysNeXt, a dual-input deep learning framework that jointly exploits video frames and STMap representations. By incorporating a spatio-temporal difference modeling unit, a cross-modal interaction module, and a structured attention-based decoder, PhysNeXt collaboratively enhances the robustness of pulse signal extraction. Experimental results demonstrate that PhysNeXt achieves more stable and fine-grained rPPG signal recovery under challenging conditions, validating the effectiveness of joint modeling of video and STMap representations. The codes will be released.

rPPG双分支网络注意力机制生命体征监测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。