首个大规模微表情数据集,助力情感计算中细微动作分析。
iMiGUE-3K: A Large-Scale Benchmark for Micro-Gesture Analysis with Self-Supervised Learning

- 构建3.4万段视频的微动作数据集,覆盖332名网球运动员
- 基于自监督学习的模型在微动作识别任务中显著提升情绪理解效果
- 适合心理诊断、人机交互等领域的研究人员使用
情感理解是情感计算与人工智能的核心挑战。现有方法多聚焦于面部表情与语音,却忽视了身体语言传递的丰富情感线索。近年来,由内在情绪驱动的无意识微动作(MGs)日益受到关注。然而,尚无支持微动作基础模型预训练的大规模数据集。为此,我们提出iMiGUE-3K基准,包含迄今最大的微动作数据集及一系列基础模型。通过模型引导的众包采集策略,构建了涵盖332位职业网球运动员过去七年公开发布会采访视频的大型数据集,共计3.4K段长视频、3700万帧,覆盖32类微动作并配有详尽标注,是首个大规模、真实场景下的细粒度手势情感分析数据集。基于此,我们提出MG-FMs——一种可迁移的手势表征学习判别性基础模型,并设立五项评估任务:微动作识别(无监督、半监督、有监督)、微动作检索与微动作情绪识别。系统评估表明,基于微动作的分析显著提升情感理解能力。本工作为微动作分析提供完整工具链,奠定心理诊断、情感计算与先进人机交互研究的基础。
原文摘要 · Abstract (English)
Emotion understanding is a fundamental challenge in affective computing and artificial intelligence. While existing approaches predominantly focus on facial expressions and speech, they often overlook the rich emotional cues conveyed through body language. Recently, micro-gestures (MGs), unintentional, subconscious movements driven by inner feelings, have attracted increasing attention as an alternative to other cues. However, there are no existing large-scale datasets supporting the pre-training of the MG foundation model. To advance MG research, we present a new benchmark for micro-gesture-based emotion understanding, featuring key contributions with a novel dataset (iMiGUE-3K) and a series of foundation models for different tasks. Using a model-based crowd-sourcing data collection strategy, we construct iMiGUE-3K, the largest MG dataset to date. It comprises video recordings from 332 distinct professional tennis players' public press interviews over the past seven years, totaling more than 3.4K long video clips and 37 million frames. The dataset includes 32 micro-gesture classes with rich descriptive annotations, making it the first large-scale, in-the-wild, video dataset for fine-grained gesture-based emotion analysis. Built on iMiGUE-3K, we propose MG-FMs, a discriminative foundation model for transferable gesture presentation learning. Based on the foundation model, we establish five comprehensive evaluation tasks: MG recognition (unsupervised, semi-supervised, supervised), MG retrieval, and MG emotion recognition. Our systematic evaluation of representative methods demonstrates that micro-gesture-based analysis significantly improves emotion understanding. We hope this work can provide comprehensive tools for MG analysis and set a solid foundation for future research in psychological diagnostics, affective computing, and advanced human-computer interaction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。