用可穿戴设备生成3D人脸点云,实现隐私保护的实时情绪识别
Towards Emotion Recognition with 3D Pointclouds Obtained from Facial Expression Images
- 通过可穿戴传感器生成3D人脸点云,替代传统2D图像
- 基于FLAME模型构建AffectNet3D数据集,支持情绪识别训练
- 仅用25%新数据微调即超越纯本地训练,适合连续监测场景
面部情绪识别在人机交互、心理健康评估和疲劳监测中具有广泛应用。当前方法多依赖2D图像数据,存在隐私风险且不适用于持续实时监测。本文提出高频无线传感(HFWS)作为替代方案,通过可穿戴设备生成精细3D人脸点云,具备更强隐私保护能力。为解决3D情绪识别数据稀缺问题,我们提出基于FLAME的3D点云生成方法,构建了AffectNet3D数据集。通过设计点云精炼流程分离面部区域,并在修正后的数据上训练PointNet++模型。在未见数据集BU-3DFE上微调,仅使用25%样本即达到70%以上分类准确率,接近理想水平。进一步模拟可穿戴条件遮蔽点云部分,结果表明:先在AffectNet3D上训练再用25% BU-3DFE微调的模型,性能优于仅在BU-3DFE上训练的模型。这验证了该流程的可行性,支持基于可穿戴HFWS系统的持续、隐私保护式情绪识别。
原文摘要 · Abstract (English)
Facial Emotion Recognition is a critical research area within Affective Computing due to its wide-ranging applications in Human Computer Interaction, mental health assessment and fatigue monitoring. Current FER methods predominantly rely on Deep Learning techniques trained on 2D image data, which pose significant privacy concerns and are unsuitable for continuous, real-time monitoring. As an alternative, we propose High-Frequency Wireless Sensing (HFWS) as an enabler of continuous, privacy-aware FER, through the generation of detailed 3D facial pointclouds via on-person sensors embedded in wearables. We present arguments supporting the privacy advantages of HFWS over traditional 2D imaging, particularly under increasingly stringent data protection regulations. A major barrier to adopting HFWS for FER is the scarcity of labeled 3D FER datasets. Towards addressing this issue, we introduce a FLAME-based method to generate 3D facial pointclouds from existing public 2D datasets. Using this approach, we create AffectNet3D, a 3D version of the AffectNet database. To evaluate the quality and usability of the generated data, we design a pointcloud refinement pipeline focused on isolating the facial region, and train the popular PointNet++ model on the refined pointclouds. Fine-tuning the model on a small subset of the unseen 3D FER dataset BU-3DFE yields a classification accuracy exceeding 70%, comparable to oracle-level performance. To further investigate the potential of HFWS-based FER for continuous monitoring, we simulate wearable sensing conditions by masking portions of the generated pointclouds. Experimental results show that models trained on AffectNet3D and fine-tuned with just 25% of BU-3DFE outperform those trained solely on BU-3DFE. These findings highlight the viability of our pipeline and support the feasibility of continuous, privacy-aware FER via wearable HFWS systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。