提出三维量化指标评估多人交互中的行为协调性,适用于数字人社交智能评测。
Multimodal Quantitative Measures for Multiparty Behaviour Evaluation
- 基于三种分析方法:交叉递归分析、多尺度节拍一致性、软动态时间规整。
- 145段30秒数据经扰动后显示:动作抑制提升确定性,语音延迟削弱耦合,音调平滑增加成本。
- 适合评估数字人交互中时空结构与行为变异性的研究者使用。
数字人作为多人交互中的自主代理正逐渐兴起,但现有评估指标大多忽略情境协调动态。本文提出一种统一的干预驱动框架,用于客观评估骨骼运动数据中的多人社会行为,涵盖三个互补维度:(1) 通过交叉递归量化分析(CRQA)衡量同步性;(2) 基于多尺度经验模态分解的节拍一致性衡量时间对齐;(3) 通过软动态时间规整(Soft DTW)衡量结构相似性。在约145段来自DnD数据集的30秒群体互动片段上,施加三种理论驱动的扰动——手势运动阻尼、均匀语音-手势延迟、语调音高方差降低——进行验证。混合效应分析揭示可预测的独立变化:阻尼使CRQA确定性上升、节拍一致性下降;延迟削弱跨参与者耦合;音调平滑提升F0 Soft-DTW成本。另开展感知研究(N=27),对比完整视频与仅骨骼渲染的判断,量化表示影响。三项指标提供关于空间结构、时间对齐与行为变异性上的正交见解,构成一套稳健的社交智能代理评估与优化工具包。代码已开源于GitHub。
原文摘要 · Abstract (English)
Digital humans are emerging as autonomous agents in multiparty interactions, yet existing evaluation metrics largely ignore contextual coordination dynamics. We introduce a unified, intervention-driven framework for objective assessment of multiparty social behaviour in skeletal motion data, spanning three complementary dimensions: (1) synchrony via Cross-Recurrence Quantification Analysis, (2) temporal alignment via Multiscale Empirical Mode Decompositionbased Beat Consistency, and (3) structural similarity via Soft Dynamic Time Warping. We validate metric sensitivity through three theory-driven perturbations -- gesture kinematic dampening, uniform speech-gesture delays, and prosodic pitch-variance reduction-applied to $\approx 145$ 30-second thin slices of group interactions from the DnD dataset. Mixed-effects analyses reveal predictable, joint-independent shifts: dampening increases CRQA determinism and reduces beat consistency, delays weaken cross-participant coupling, and pitch flattening elevates F0 Soft-DTW costs. A complementary perception study ($N=27$) compares judgments of full-video and skeleton-only renderings to quantify representation effects. Our three measures deliver orthogonal insights into spatial structure, timing alignment, and behavioural variability. Thereby forming a robust toolkit for evaluating and refining socially intelligent agents. Code available on \href{https://github.com/tapri-lab/gig-interveners}{GitHub}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。