arXiv:2609.06928cs.CVcs.AI2026-09

构建首个大规模事件相机情绪识别数据集,提升隐私保护下的情绪分析性能。

Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras

论文配图:Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras
图 1 · 摘自论文原文
  • 提出多模态融合框架IGF,结合事件、音频与文本信息。
  • 在Emo-DVS数据集上达到当前最优效果,准确率显著领先。
  • 适合关注隐私安全与多模态感知的研究者使用。

情绪分析是计算机视觉的基础任务,但传统RGB摄像头存在隐私泄露风险。生物启发的事件相机通过捕捉异步亮度变化,在降低面部身份暴露的同时,利用高动态范围实现复杂光照下的鲁棒感知。然而,现有事件基方法在真实场景中表现受限,主要因数据规模小、采集条件简单且依赖单一视觉模态。为此,我们构建了包含事件、音频和文本三模态的挑战性基准Emo-DVS,提出信息引导门控融合(IGF)框架:首先在Emo-DVS的FAU子集上预训练事件编码器以捕获细微面部动作,再通过自适应模态门控抑制噪声,最后利用互信息最大化对齐跨模态表示。为缓解数据稀缺问题,我们推出了首个大规模事件相机情绪分析数据集Emo-DVS,融合动态光照与面部动作单元(FAU)及情绪子集。大量实验表明,IGF达到当前最佳性能。

原文摘要 · Abstract (English)

Emotion analysis is a fundamental task in computer vision, but its practical deployment remains constrained by the privacy risks inherent to conventional RGB cameras. Bio-inspired event cameras present a promising hardware-level solution because they capture asynchronous brightness changes, thereby reducing exposure of facial identity details while leveraging high dynamic range for robust perception under challenging illumination conditions. Despite these advantages, existing event-based methods struggle in complex real-world settings due to limited dataset scales, simple acquisition conditions, and reliance on single-modality visual cues. To address these, we establish a challenging tri-modal benchmark with event, audio, and text modalities and propose the Information-Guided Gated Fusion (IGF) framework, which first pre-trains an event encoder on the FAU subset of Emo-DVS to capture fine-grained facial dynamics, then employs adaptive modality gating to suppress modality-specific noise, and finally leverages mutual information maximization to align robust cross-modal representations. To alleviate data scarcity, we introduce Emo-DVS, the first large-scale event-based emotion analysis dataset, which couples dynamic illumination with the Facial Action Unit (FAU) subset and emotion subset. Extensive experiments demonstrate that IGF achieves state-of-the-art performance.

情绪识别事件相机多模态隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。