用快速傅里叶加速的托普利茨结构,实现轻量级远程脉搏波估计。
ToTMNet: FFT-Accelerated Toeplitz Temporal Mixing Network for Lightweight Remote Photoplethysmography
- 用托普利茨矩阵替代注意力,线性参数量捕捉长序列时序依赖。
- 在真实视频上达到1.055 bpm误差,相关系数达0.996。
- 仅63k参数,适合移动端部署,尤其适应跨域场景。
远程光体积描记法(rPPG)通过普通摄像头拍摄的面部视频估计血容量脉搏波形。尽管近期深度模型相比传统信号处理方法提升了鲁棒性,但多数方法增加了计算开销和参数量,基于注意力的时序建模还存在与序列长度平方相关的复杂度。本文提出ToTMNet,一种轻量级rPPG架构,用快速傅里叶加速的托普利茨时序混合层替代时序注意力。托普利茨算子以线性数量参数实现全序列时序感受野,并通过循环嵌入和基于FFT的卷积实现在近线性时间内运算。ToTMNet将全局托普利茨算子集成到紧凑的门控时序混合器中,结合局部深度可分离时序卷积分支与门控全局托普利茨混合,实现高效长程时序滤波,总参数仅63k。在两个数据集上实验表明:在UBFC-rPPG(真实视频)上,内数据集评估达1.055 bpm MAE,皮尔逊相关系数0.996;在合成到真实设置(SCAMPS→UBFC-rPPG)下,达1.582 bpm MAE,相关系数0.994。消融实验确认门控机制对有效利用全局托普利茨混合至关重要,尤其在域偏移下。该预印本研究主要局限在于仅使用两个数据集,但结果表明托普利茨结构时序混合是rPPG中注意力的有效且高效的替代方案。
原文摘要 · Abstract (English)
Remote photoplethysmography (rPPG) estimates a blood volume pulse (BVP) waveform from facial videos captured by commodity cameras. Although recent deep models improve robustness compared to classical signal-processing approaches, many methods increase computational cost and parameter count, and attention-based temporal modeling introduces quadratic scaling with respect to the temporal length. This paper proposes ToTMNet, a lightweight rPPG architecture that replaces temporal attention with an FFT-accelerated Toeplitz temporal mixing layer. The Toeplitz operator provides full-sequence temporal receptive field using a linear number of parameters in the clip length and can be applied in near-linear time using circulant embedding and FFT-based convolution. ToTMNet integrates the global Toeplitz temporal operator into a compact gated temporal mixer that combines a local depthwise temporal convolution branch with gated global Toeplitz mixing, enabling efficient long-range temporal filtering while only having 63k parameters. Experiments on two datasets, UBFC-rPPG (real videos) and SCAMPS (synthetic videos), show that ToTMNet achieves strong heart-rate estimation accuracy with a compact design. On UBFC-rPPG intra-dataset evaluation, ToTMNet reaches 1.055 bpm MAE with Pearson correlation 0.996. In a synthetic-to-real setting (SCAMPS to UBFC-rPPG), ToTMNet reaches 1.582 bpm MAE with Pearson correlation 0.994. Ablation results confirm that the gating mechanism is important for effectively using global Toeplitz mixing, especially under domain shift. The main limitation of this preprint study is the use of only two datasets; nevertheless, the results indicate that Toeplitz-structured temporal mixing is a practical and efficient alternative to attention for rPPG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。