arXiv:2411.01542cs.CV2024-11NeurIPS被引 27

提出多维注意力分解方法,提升远程生理信号提取精度

FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing

  • 用非负矩阵分解联合建模时空通道注意力
  • 在4个公开数据集上显著优于现有方法
  • 适合需要多维特征融合的生理信号研究者

远程光电容积脉搏波(rPPG)通过成像实现无创血容量脉搏信号提取,将时空数据转化为时间序列信号。当前端到端rPPG方法聚焦于这一转换过程,注意力机制在特征提取中至关重要。然而,现有方法在空间、时间与通道维度上独立计算注意力。本文提出因子化自注意力模块(FSAM),通过非负矩阵分解从体素嵌入中联合计算多维注意力。为验证其有效性,构建了基于3D-CNN的FactorizePhys架构,用于从原始视频帧中估计血容量脉搏信号。该方法有效分解体素嵌入,实现全面的空间、时间与通道注意力,提升通用信号提取性能。此外,将FSAM集成至现有2D-CNN基rPPG架构中,展示其通用性。在四个公开数据集上评估并对比先进rPPG方法,涵盖不同架构与注意力机制类型。通过消融实验分析架构设计与超参数选择。可视化学习到的时空特征及跨数据集泛化能力证实了FSAM的有效性,表明其作为多维注意力机制的广泛应用潜力。代码开源:https://github.com/PhysiologicAILab/FactorizePhys。

原文摘要 · Abstract (English)

Remote photoplethysmography (rPPG) enables non-invasive extraction of blood volume pulse signals through imaging, transforming spatial-temporal data into time series signals. Advances in end-to-end rPPG approaches have focused on this transformation where attention mechanisms are crucial for feature extraction. However, existing methods compute attention disjointly across spatial, temporal, and channel dimensions. Here, we propose the Factorized Self-Attention Module (FSAM), which jointly computes multidimensional attention from voxel embeddings using nonnegative matrix factorization. To demonstrate FSAM's effectiveness, we developed FactorizePhys, an end-to-end 3D-CNN architecture for estimating blood volume pulse signals from raw video frames. Our approach adeptly factorizes voxel embeddings to achieve comprehensive spatial, temporal, and channel attention, enhancing performance of generic signal extraction tasks. Furthermore, we deploy FSAM within an existing 2D-CNN-based rPPG architecture to illustrate its versatility. FSAM and FactorizePhys are thoroughly evaluated against state-of-the-art rPPG methods, each representing different types of architecture and attention mechanism. We perform ablation studies to investigate the architectural decisions and hyperparameters of FSAM. Experiments on four publicly available datasets and intuitive visualization of learned spatial-temporal features substantiate the effectiveness of FSAM and enhanced cross-dataset generalization in estimating rPPG signals, suggesting its broader potential as a multidimensional attention mechanism. The code is accessible at https://github.com/PhysiologicAILab/FactorizePhys.

生理信号注意力机制视频分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。