用雷达数据训练视频模型,让其在光照变化下仍能准确测心率呼吸。
RPM-Distill: Physiology-guided Adaptive Cross-modal Distillation for Robust Remote Physiological Measurement

- 用频域周期性特征实现视频与雷达跨模态知识蒸馏
- 在复杂环境下比单模态方法误差降低81%,相关性提升21%
- 自适应加权蒸馏,适合无雷达设备的部署场景
基于视频的远程生理测量(RPM)虽便捷,但在光照、肤色和运动变化下易失效。射频(RF)雷达对光照和外观不敏感,可提供互补的心肺微动信号,但推理时需雷达设备常不现实。本文提出RPM-Distill,仅在训练阶段使用同步雷达,推理阶段仅需视频。核心观察是:尽管RGB与RF波形在传感原理和时域形态上不同,但其频域具有相似的潜在周期性。因此,通过损失函数(1)锚定基频峰值,(2)匹配非峰背景分布,(3)保持频谱形态与锐度,实现生理结构化的频域证据蒸馏。为避免样本级教师质量与对齐不确定性导致负迁移,引入频谱策略网络,从学生-教师频谱关系图中预测样本级蒸馏门控与组件权重,采用小规模标注验证集上的元双层优化目标进行学习。在挑战性条件和跨数据集设置下,RPM-Distill相比单模态基线实现81%的平均绝对误差降低和21%的相关性提升。代码已开源。
原文摘要 · Abstract (English)
Video-based remote physiological measurement (RPM) is highly accessible but remains fragile under varying illumination, skin tones, and motion. Radio frequency (RF) radar is largely invariant to illumination and appearance, providing complementary cardio-respiratory micro-motion cues; however, requiring radar at inference is often impractical due to its limited ubiquity and deployment overhead. We propose RPM-Distill, a physiology-guided cross-modal distillation framework that leverages synchronized radar only during training while retaining video-only inference. Our key observation is that although RGB and RF waveforms differ in sensing physics and time-domain morphology, they share similar latent periodic rhythm in the frequency domain. We thus distill physiology-structured spectral evidence to improve robustness, via losses that (i) anchor the fundamental peak, (ii) match the off-peak background distribution, and (iii) preserve spectral morphology and sharpness. To avoid negative transfer under sample-level teacher quality and alignment uncertainty, a spectral policy network predicts sample-level distillation gates and component weights from the student--teacher spectral relation map, learned with a meta bilevel objective on a small labeled validation split. Through extensive experiments in challenging conditions and cross-dataset settings, RPM-Distill brings 81\% MAE and 21\% correlation improvement over unimodal baselines. Code is at https://github.com/WJULYW/RPM-Distill.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。