通过头戴摄像头实时估算心率变异性，实现对压力与专注度的连续感知。
EgoHRV: Continuous Heart Rate Variability Estimation from Egocentric Systems for Autonomic Response and Skill Assessment
- 利用3D骨干网络和高低频分解模块，从注视视频中提取微弱血容量脉冲信号
- 跨域预训练使摄像头信号与接触式信号在频域对齐，实现毫秒级时间精度
- 支持压力、注意力等心理状态评估，适合人机交互与技能评测场景
头戴视觉系统可捕捉人类行为的可见线索，但忽略了压力、投入度和注意力等自主神经状态的生理指标。心率变异性（HRV）是评估应激下自主调节能力的非侵入性指标，反映相邻心跳间微小的时间差异。然而，由于注视视频中的运动与噪声干扰，传统头戴设备难以获取此类精细时序信息。本文提出EgoHRV，一种从头戴摄像头视频中同时估计心率（HR）与心率变异性（HRV）的方法。其流水线结合3D骨干网络与新型低-高频分解模块，从注视视频中提取血容量脉冲（BVP）信号。通过跨域预训练对齐接触式信号与相机信号的频域表示，使EgoHRV具备恢复微小波动的时序精度。在真实数据上，该方法在头戴视频中实现了当前最优的HR与HRV估计性能，其不确定性感知设计提升了下游行为建模效果。将HRV估计与置信度引入EgoExo4D技能评估器后，准确率提升17.8%。该技术为头戴系统开启压力与唤醒度感知任务提供了可能。
原文摘要 · Abstract (English)
Egocentric vision systems capture human behavior from visible cues, but overlook physiological indicators of autonomic states such as stress, engagement, and attention. Heart rate variability (HRV) is a widely used noninvasive marker of autonomic regulation under stress. HRV reflects small timing differences between successive heartbeats and has so far been out of reach for egocentric platforms, where motion and noise in gaze video mask exactly this fine-grained timing. We propose EgoHRV, a method that estimates HRV as well as heart rate (HR) from the gaze cameras that are already integrated into egocentric headsets. Our pipeline combines a 3D backbone with a novel low--high decomposition module that extracts the blood volume pulse (BVP) signal from gaze video. Our cross-domain pretraining aligns the frequency-domain representations of contact-based and camera-derived signals. This alignment gives EgoHRV the temporal precision to recover HRV from the subtle fluctuations in gaze video. EgoHRV achieves state-of-the-art accuracy for HR and HRV estimation from egocentric video, and its uncertainty-aware design improves downstream behavioral modeling. Integrating our HRV estimates and confidence measures into EgoExo4D's proficiency estimator raises accuracy by 17.8%. Beyond skill, continuous HRV estimation also opens egocentric systems to stress- and arousal-aware estimation tasks. Code: https://github.com/eth-siplab/EgoHRV
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。