通用自监督模型难捕捉主观情绪,个性化微调才是关键。
Take it Personally: The Limits of General SSL Representations for Real-Life PPG Emotion Detection

- 用真实生活数据预训练PPG模型,提升运动识别性能
- 在个体情绪识别中,通用表示效果远低于基础模型
- 加入个人数据微调显著提升效果,适合真实场景研究
尽管自监督学习(SSL)能从噪声大、无约束的生理信号(如光电容积脉搏波描记法,PPG)中提取通用表征,但其在高度主观任务中的适用性尚未验证。本文评估了基于PPG的SSL在真实生活强烈情绪检测中的有效性。首先,我们在真实生活、无约束数据上预训练了一个实时PPG编码器(RL-PPG)。作为严谨的验证,我们证明这些表征在客观运动识别任务中表现优异,相较于基线在留一被试者外(LOSO)评估中性能提升近5倍。然而,在主观的真实生活情绪检测任务中,相同通用表征在相同评估协议下仍无法超越朴素基线。通过跨时间验证策略,我们发现个体个人数据的微调是预测性能的主要驱动力,远超群体级预训练带来的收益。最终结果表明,在所评估场景中,通用SSL表征可能不足以支持主观情感推断,提示个性化可能是真实世界情绪识别的关键。为支持后续研究,我们公开代码和预训练的RL-PPG编码器权重。
原文摘要 · Abstract (English)
While Self-Supervised Learning (SSL) effectively extracts general representations from noisy, unconstrained physiological signals such as photoplethysmography (PPG), its suitability for highly subjective tasks remains unproven. In this work, we evaluate the efficacy of PPG-based SSL for real-life intense emotion detection. First, we pretrain a Real-Life PPG encoder (RL-PPG) on unconstrained, real-life data. As a rigorous sanity check, we demonstrate that these representations transfer exceptionally well to an objective physical activity recognition task, yielding almost 5-fold increase in performance over baselines in a leave-one-subject-out evaluation (LOSO). However, when applied to a~subjective real-life emotion detection task, these same general representations fail to surpass naive baselines under the LOSO protocol. Using an Across-Time validation strategy, we establish that incorporating an individual's personal data during fine-tuning is the main driver of predictive performance, outweighing the benefits of population-level pretraining. Ultimately, our findings indicate that in the evaluated scenario, general SSL representations may be insufficient for subjective affective inference, suggesting that personalization is likely a key component for real-world emotion recognition. To support future research, we share the code and pretrained RL-PPG~encoder~weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。