用情感增强生成技术解决动态表情识别数据少难题
ARGen: Affect-Reinforced Generative Augmentation towards Vision-based Dynamic Emotion Perception

- 通过情感语义注入和强化扩散,自适应生成高质量表情视频
- 在多个数据集上提升表情识别准确率,显著改善罕见情绪识别效果
- 适合做情感计算、视频生成与数据增强的研究者参考
野外动态面部表情识别因数据稀缺和长尾分布仍具挑战,限制了模型对稀有情绪时序动态的学习。为此,我们提出 ARGen——一种情感增强生成增强框架,实现面向鲁棒情感感知的数据自适应动态表情生成。该框架分两阶段运行:情感语义注入(ASI)阶段通过面部动作单元建立情感知识对齐,并利用检索增强提示生成策略,借助大规模视觉-语言模型合成一致且细粒度的情感描述,将可解释的情绪先验注入生成过程;自适应强化扩散(ARD)阶段融合文本条件图像到视频扩散模型与强化学习,引入帧间条件引导及多目标奖励函数,联合优化表情自然性、面部完整性与生成效率。大量生成与识别任务实验表明,ARGen显著提升生成保真度并改善识别性能,构建了一个可解释、通用的视觉情感计算生成增强范式。
原文摘要 · Abstract (English)
Dynamic facial expression recognition in the wild remains challenging due to data scarcity and long-tail distributions, which hinder models from effectively learning the temporal dynamics of scarce emotions. To address these limitations, we propose ARGen, an Affect-Reinforced Generative Augmentation Framework that enables data-adaptive dynamic expression generation for robust emotion perception. ARGen operates in two stages: Affective Semantic Injection (ASI) and Adaptive Reinforcement Diffusion (ARD). The ASI stage establishes affective knowledge alignment through facial Action Units and employs a retrieval-augmented prompt generation strategy to synthesize consistent and fine-grained affective descriptions via large-scale visual-language models, thereby injecting interpretable emotional priors into the generation process. The ARD stage integrates text-conditioned image-to-video diffusion with reinforcement learning, introducing inter-frame conditional guidance and a multi-objective reward function to jointly optimize expression naturalness, facial integrity, and generative efficiency. Extensive experiments on both generation and recognition tasks verify that ARGen substantially enhances synthesis fidelity and improves recognition performance, establishing an interpretable and generalizable generative augmentation paradigm for vision-based affective computing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。