arXiv:2606.31736cs.CV2026-06

通过节奏结构建模提升远程脉搏波信号估计精度

Rhythm-Structured Predictive Learning for Remote Photoplethysmography

论文配图:Rhythm-Structured Predictive Learning for Remote Photoplethysmography
图 1 · 摘自论文原文
  • 用隐状态预测替代像素重建,让模型关注生理动态
  • 设计循环节律规划器,精准捕捉脉搏的周期性特征
  • 适合需要高精度生理信号提取的研究者和开发者

远程光电容积脉搏波(rPPG)通过分析面部视频中由脉搏引起的微弱肤色变化来估计生理信号。尽管近期取得进展,现有自监督rPPG方法主要依赖于掩码像素或低层视觉表示的重建,易使模型偏向面部外观而非潜在生理动力学。此外,大多数基于Mamba的方法仅按时间顺序扫描面部视频标记,限制了对脉搏信号周期结构的利用。为此,我们提出RhythmJEPA,一种节奏结构化的联合嵌入预测学习框架。不同于重建RGB帧,RhythmJEPA从掩码面部视频中预测隐教师表示,从而在嵌入空间中促进生理感知表征学习。为显式建模脉搏相关的时间结构,引入循环节律状态规划器(CRSP),通过动态规划与约束转移语法解码最合理的周期状态路径。基于解码状态,设计双序Mamba编码器(DOM),结合常规时间扫描与状态有序扫描,以捕捉局部时序连续性和长程节奏一致性依赖。最后,轻量级空间脉冲混合器(SPM)提取紧凑的脉冲敏感面部标记,在复杂度与性能间取得良好平衡。在PURE、UBFC-rPPG和MMPD数据集上的实验表明,该方法优于代表性rPPG方法。代码已开源:https://github.com/deconasser/RhythmJEPA。

原文摘要 · Abstract (English)

Remote photoplethysmography (rPPG) estimates physiological signals from facial videos by analyzing subtle pulse induced skin color variations. Despite recent progress, existing self-supervised rPPG methods mainly reconstruct masked pixels or low-level visual representations, which can bias the model toward facial appearance rather than latent physiological dy namics. Moreover, most recent Mamba-based approaches scan facial video tokens only in chronological order, limiting their ability to exploit the cyclic structure of pulse signals. To ad dress these limitations, we propose RhythmJEPA, a rhythm structured joint-embedding predictive learning framework for rPPG. Instead of reconstructing RGB frames, RhythmJEPA predicts latent teacher representations from masked facial videos, thereby encouraging physiology-aware representation learning in the embedding space. To explicitly model pulse-related tem poral structure, we introduce a Cyclic Rhythm-State Plan ner (CRSP), which estimates frame-wise latent physiological states and decodes the most plausible cyclic state path via dynamic programming with a constrained transition grammar. Guided by the decoded states, we further design a Dual Order Mamba Encoder (DOM), which combines conventional chronological scanning with state-ordered scanning to capture both local temporal continuity and long-range rhythm-consistent dependencies. Finally, a lightweight Spatial Pulse Mixer (SPM) extracts compact pulse-sensitive facial tokens with a favorable balance between complexity and performance. Experiments on PURE, UBFC-rPPG, and MMPD show competitive performance over representative rPPG methods. The codes are available at https://github.com/deconasser/RhythmJEPA.

rPPG脉搏检测Mamba自监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。