arXiv:2510.17855cs.CV2025-10

通过多尺度归一化,让AI更懂每个人表达同意时的独特习惯。

CMIS-Net: A Cascaded Multi-Scale Individual Standardization Network for Backchannel Agreement Estimation

  • 分帧和序列双尺度建模个体差异,提取相对变化特征。
  • 在CMU-MOSEI数据集上达到89.3%准确率,优于现有方法。
  • 适合做对话系统、人机交互中情感理解的开发者使用。

后通道信号(如点头、微笑或'是'等简短回应)是对话中体现理解与认同的微妙反馈,对提升人机交互流畅性至关重要。然而,这些行为受个体差异显著影响,涉及多个层次:从单帧响应强度(帧级)到频率与节奏偏好(序列级)。现有情感识别方法多仅关注单一尺度,忽略多尺度线索的互补性。为此,本文提出级联多尺度个体标准化网络(CMIS-Net),通过去除个体特有基线值,实现表达特征的个体归一化,使模型聚焦于相对于个人基准的变化。同时引入隐式数据增强模块,缓解训练数据分布偏差,提升泛化能力。大量实验与可视化表明,该方法有效处理个体差异与数据不平衡问题,在后通道认同检测任务中取得领先性能。

原文摘要 · Abstract (English)

Backchannels are subtle listener responses, such as nods, smiles, or short verbal cues like "yes" or "uh-huh," which convey understanding and agreement in conversations. These signals provide feedback to speakers, improve the smoothness of interaction, and play a crucial role in developing human-like, responsive AI systems. However, the expression of backchannel behaviors is often significantly influenced by individual differences, operating across multiple scales: from instant dynamics such as response intensity (frame-level) to temporal patterns such as frequency and rhythm preferences (sequence-level). This presents a complex pattern recognition problem that contemporary emotion recognition methods have yet to fully address. Particularly, existing individualized methods in emotion recognition often operate at a single scale, overlooking the complementary nature of multi-scale behavioral cues. To address these challenges, we propose a novel Cascaded Multi-Scale Individual Standardization Network (CMIS-Net) that extracts individual-normalized backchannel features by removing person-specific neutral baselines from observed expressions. Operating at both frame and sequence levels, this normalization allows model to focus on relative changes from each person's baseline rather than absolute expression values. Furthermore, we introduce an implicit data augmentation module to address the observed training data distributional bias, improving model generalization. Comprehensive experiments and visualizations demonstrate that CMIS-Net effectively handles individual differences and data imbalance, achieving state-of-the-art performance in backchannel agreement detection.

情感识别多尺度建模个性化建模对话系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。