arXiv:2601.00714eess.IV2026-01被引 4

用轻量模型实时提取视频中的生理信号,精度更高且速度快一倍。

KDPhys: An Attention Guided 3D to 2D Knowledge Distillation for Real-time Video-Based Physiological Measurement

  • 用3D CNN教2D CNN学,通过注意力机制实现跨维度特征蒸馏。
  • 参数仅0.23M,速度提升56.67%,平均误差点搏数1.78 bpm。
  • 适合部署在移动端或嵌入式设备,抗干扰能力强。

基于摄像头的生理监测(如远程光电容积脉搏波描记术,rPPG)利用普通数码相机捕捉皮肤光学特性随血容量波动产生的微小变化。随着新冠疫情推动远程健康监测需求激增,对实时、非接触式生理测量的需求显著上升。本文提出一种基于注意力的知识蒸馏框架KDPhys,从面部视频序列中提取rPPG信号。该方法将3D卷积神经网络教师模型的全局时序特征,通过高效的3D到2D特征蒸馏方式传递给轻量级2D卷积神经网络学生模型。据我们所知,这是首次将知识蒸馏应用于rPPG领域。此外,引入包含形态与时间特性的畸变损失(DILATE),联合建模rPPG信号的形貌与时间动态。在三个基准数据集上进行了广泛评估,结果表明,该模型计算复杂度显著降低:参数量仅为现有方法的一半,运行速度提升56.67%。仅使用0.23M参数,相比最先进方法,平均绝对误差(MAE)降低18.15%,跨数据集平均MAE达1.78 bpm。在多种环境条件和活动场景下的额外实验进一步验证了方法的鲁棒性与适应性。

原文摘要 · Abstract (English)

Camera-based physiological monitoring, such as remote photoplethysmography (rPPG), captures subtle variations in skin optical properties caused by pulsatile blood volume changes using standard digital camera sensors. The demand for real-time, non-contact physiological measurement has increased significantly, particularly during the SARS-CoV-2 pandemic, to support telehealth and remote health monitoring applications. In this work, we propose an attention-based knowledge distillation (KD) framework, termed KDPhys, for extracting rPPG signals from facial video sequences. The proposed method distills global temporal representations from a 3D convolutional neural network (CNN) teacher model to a lightweight 2D CNN student model through effective 3D-to-2D feature distillation. To the best of our knowledge, this is the first application of knowledge distillation in the rPPG domain. Furthermore, we introduce a Distortion Loss incorporating Shape and Time (DILATE), which jointly accounts for both morphological and temporal characteristics of rPPG signals. Extensive qualitative and quantitative evaluations are conducted on three benchmark datasets. The proposed model achieves a significant reduction in computational complexity, using only half the parameters of existing methods while operating 56.67% faster. With just 0.23M parameters, it achieves an 18.15% reduction in Mean Absolute Error (MAE) compared to state-of-the-art approaches, attaining an average MAE of 1.78 bpm across all datasets. Additional experiments under diverse environmental conditions and activity scenarios further demonstrate the robustness and adaptability of the proposed approach.

视频生理知识蒸馏rPPG轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。