用3D人脸网格建模颜色变化，提升远程测心率精度
Facial Spatiotemporal Graphs: Leveraging the 3D Facial Surface for Remote Physiological Measurement
- 构建3D人脸时序图，对齐面部表面结构进行处理
- 在4个数据集上性能领先，跨数据集泛化能力强
- 适合做可解释的生理信号监测系统的研究者
面部远程光电容积脉搏波描记法（rPPG）通过建模3D人脸表面随时间的微弱颜色变化来估计生理信号。现有方法未能显式对齐其感受野与3D人脸表面——rPPG信号的空间支撑。为此，我们提出面部时空图（STGraph），一种利用3D人脸网格序列编码面部颜色与结构的新表示，实现表面对齐的时空处理。我们设计MeshPhys，一个轻量级时空图卷积网络，在STGraph上估计生理信号。在四个基准数据集上，MeshPhys在跨数据集和内部数据集设置中均达到当前最优或竞争性表现。消融实验表明，将模型感受野限制在面部表面可作为强结构先验，且表面对齐的3D感知节点特征对鲁棒编码面部颜色至关重要。STGraph与MeshPhys共同构成一种新颖、合理的面部rPPG建模范式，实现鲁棒、可解释、可泛化的生理信号估计。代码已公开于https://samcantrill.github.io/facial-stgraph-rppg/。
原文摘要 · Abstract (English)
Facial remote photoplethysmography (rPPG) methods estimate physiological signals by modeling subtle color changes on the 3D facial surface over time. However, existing methods fail to explicitly align their receptive fields with the 3D facial surface-the spatial support of the rPPG signal. To address this, we propose the Facial Spatiotemporal Graph (STGraph), a novel representation that encodes facial color and structure using 3D facial mesh sequences-enabling surface-aligned spatiotemporal processing. We introduce MeshPhys, a lightweight spatiotemporal graph convolutional network that operates on the STGraph to estimate physiological signals. Across four benchmark datasets, MeshPhys achieves state-of-the-art or competitive performance in both intra- and cross-dataset settings. Ablation studies show that constraining the model's receptive field to the facial surface acts as a strong structural prior, and that surface-aligned, 3D-aware node features are critical for robustly encoding facial surface color. Together, the STGraph and MeshPhys constitute a novel, principled modeling paradigm for facial rPPG, enabling robust, interpretable, and generalizable estimation. Code is available at https://samcantrill.github.io/facial-stgraph-rppg/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。