融合视频与事件流，非接触精准还原脉搏波形细节。
Fusion-E2Pulse: A Multimodal Event-RGB Fusion Network for Non-contact Pulse Wave Reconstruction
- 用滤波后视频作结构先验，事件流捕捉微弱脉动。
- 心率误差仅0.78 bpm，波形相关性达0.89，相位误差16.74 ms。
- 适合需要高精度脉搏分析的医疗健康场景。
非接触式脉搏波重建依赖于波形形态的精确恢复，包括重搏切迹等细微特征。传统基于红绿蓝（RGB）的方法从面部视频中提取生理信号，受限于标准相机的积分成像机制，曝光过程导致微弱血管搏动细节被平滑削弱。相反，类脑事件相机虽对亮度变化高度敏感，但易受微小运动引起的噪声和伪影干扰。为发挥帧图像整合与事件差分感知的协同优势，本文提出新型多模态网络Fusion-E2Pulse。该框架以滤波后的RGB信号作为结构先验，抑制运动伪影，同时利用事件流的高灵敏度恢复细粒度波形细节。实验表明，Fusion-E2Pulse达到当前最优性能，有效平衡噪声抑制与形态保真度，心率估计均方误差为0.78 bpm，波形相关性达0.89，收缩相持续时间误差为16.74 ms，验证了其在重建细微病理特征方面的有效性。
原文摘要 · Abstract (English)
Non-contact pulse wave reconstruction hinges on the precise recovery of waveform morphology, including the dicrotic notch. Conventional Red-Green-Blue (RGB)-based methods, which extract physiological signals from recorded facial videos, are constrained by the integral imaging mechanism of standard cameras, where the exposure process induces a smoothing effect that attenuates subtle vascular pulsation details. Conversely, neuromorphic event cameras, while offering exceptional sensitivity to intensity fluctuations, are inherently susceptible to noise and artifacts induced by minor motion. To exploit the synergy between frame-based integration and event-based differential sensing, we propose a novel multimodal network named Fusion-E2Pulse. This framework utilizes filtered RGB signals as structural priors to suppress motion artifacts, while leveraging the high-sensitivity of event streams to recover fine-grained morphological details. Experimental results demonstrate that Fusion-E2Pulse achieves state-of-the-art performance, effectively balancing noise suppression and morphological fidelity, achieving a mean absolute error of 0.78 bpm for heart rate estimation, a waveform correlation of 0.89, and a systolic phase duration error of 16.74 ms, validating its efficacy in reconstructing fine-grained pathological features.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。