arXiv:2607.16285cs.CV2026-07被引 1

用共享隐变量统一处理部分标注的面部情绪多任务识别。

A Shared Latent for Partially-Labeled Multi-Task Facial Affect Recognition

论文配图:A Shared Latent for Partially-Labeled Multi-Task Facial Affect Recognition
图 1 · 摘自论文原文
  • 构建共享隐变量,让不同任务间通过变分瓶颈传递信号。
  • 在仅37%数据全标注时,表情识别宏F1提升至0.446。
  • 适用于标注不全、类别不平衡的真实场景,适合实际部署。

真实世界中的面部情绪天然具有多任务特性:效价-唤醒度、离散表情和面部动作单元描述同一张脸。然而真实数据集仅部分标注这些任务且分布不均,多数系统会忽略缺失标签或伪补,从而丢失跨任务信息。本文将部分标注的多任务学习建模为共享情绪隐变量的边际化:一个变分瓶颈同时服务三个任务解码器,使得有标注任务的信号能影响其他任务的表示,而被遮蔽的目标函数则转化为证据下界中的重建项。在s-Aff-Wild2数据集上(仅37%帧拥有全部三类标签,类别严重不平衡,禁止使用源数据预训练),我们验证了该方法的有效性。仅用单一主干网络,表情识别宏F1从专用模型的0.403提升至0.446,而遮蔽损失模型无法达到;引入第二个误差解耦的主干网络后,突破了外部动作单元数据无法逾越的动作单元识别上限,而效价-唤醒度性能保持在噪声水平。所有增益均有对照实验验证,表明稀有类别失败源于表征问题,而非损失设计。由于各任务的源数据在测试集上独立选择,我们报告组合多任务得分1.679作为样本内终点,并基于受控比较得出结论;表达优势在AffectNet和RAF-DB上的微弱迁移被视作探索性发现,非定论。

原文摘要 · Abstract (English)

Facial affect in the wild is naturally multi-task: valence-arousal, discrete expressions, and facial action units describe the same face. Yet real corpora annotate these tasks only partially and unevenly, so most systems mask the missing labels or impute pseudo-labels and forgo the cross-task signal. We instead cast partially-labeled multi-task learning as marginalization over a shared affect latent: one variational bottleneck mediates all three task decoders, so a frame annotated for one task shapes the representation the others use, and the masked objective reappears as the reconstruction term of an evidence lower bound. On s-Aff-Wild2, where only 37% of frames carry all three labels, the classes are severely imbalanced, and pretraining on the source data is disallowed, we isolate where this coupling acts. On a single backbone it lifts expression macro-F1 from 0.403 for a dedicated specialist to 0.446, which the masked-loss model does not reach; a second, near-peer backbone with decorrelated errors then breaks an action-unit ceiling that external action-unit data could not, while valence-arousal stays within noise. Every gain is disciplined by a matched-control negative; together these controls indicate that the rare-class failure is representational, not a matter of loss shaping. As each task's source is chosen on the evaluation split, we report the assembled result, a combined multi-task score of 1.679 on validation, as an in-sample endpoint and rest our conclusions on the controlled comparisons; a small, regime-dependent transfer of the expression advantage to AffectNet and RAF-DB is presented as exploratory rather than conclusive.

多任务学习情绪识别隐变量建模部分标注

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。