用特权信息训练情感模型，让实验室效果更好迁移到真实场景。
Privileged Contrastive Pretraining for Multimodal Affect Modelling
- 先用监督对比学习预训练，再在特权信息框架中优化模型
- 在RECOLA和AGAIN数据集上超越传统方法，接近全模态表现
- 适合需要真实环境鲁棒性的跨场景情感分析任务
情感计算（AC）因深度学习取得显著进展，但核心挑战仍存：如何将实验室环境中的情感模型可靠迁移至真实世界。为此，我们提出特权对比预训练（PriCon）框架——模型先通过监督对比学习（SCL）预训练，再作为教师模型参与学习使用特权信息（LUPI）的机制。该方法在训练中利用特权信息，并通过SCL增强模型鲁棒性。在两个基准情感语料库RECOLA和AGAIN上的实验表明，PriCon模型持续优于传统LUPI和端到端模型。尤其值得注意的是，在多数情况下，其性能接近于同时拥有所有模态信息的模型在训练和测试阶段的表现。结果表明，PriCon有望成为缩小实验室与真实场景间情感建模差距的新范式，为实际应用提供可扩展、实用的解决方案。
原文摘要 · Abstract (English)
Affective Computing (AC) has made significant progress with the advent of deep learning, yet a persistent challenge remains: the reliable transfer of affective models from controlled laboratory settings (in-vitro) to uncontrolled real-world environments (in-vivo). To address this challenge we introduce the Privileged Contrastive Pretraining (PriCon) framework according to which models are first pretrained via supervised contrastive learning (SCL) and then act as teacher models within a Learning Using Privileged Information (LUPI) framework. PriCon both leverages privileged information during training and enhances the robustness of derived affect models via SCL. Experiments conducted on two benchmark affective corpora, RECOLA and AGAIN, demonstrate that models trained using PriCon consistently outperform LUPI and end to end models. Remarkably, in many cases, PriCon models achieve performance comparable to models trained with access to all modalities during both training and testing. The findings underscore the potential of PriCon as a paradigm towards further bridging the gap between in-vitro and in-vivo affective modelling, offering a scalable and practical solution for real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。