arXiv:2508.07839cs.RO2025-08被引 2

机器人用触觉和声音结合传递情感,效果比单独使用更好。

Touch Speaks, Sound Feels: A Multimodal Approach to Affective and Social Touch from Robots to Humans

  • 用震动电机与音频同步,实现触觉-听觉多模态刺激。
  • 触觉加声音组合的识别准确率显著高于单一模式。
  • 适合研究人机情感交互、社交机器人设计者阅读。

情感触觉交互是人类沟通的基础。自然的人类互动中,触觉很少孤立存在,而是与伴随的听觉线索共同作用。触觉与听觉信息的融合构成了一种丰富而细腻的情感表达通道。尽管已有大量研究探讨机器人通过面部表情和语音传达情绪,但其通过触觉传递社交手势与情绪的能力仍鲜有探索。为此,我们开发了一套多模态交互系统,包含5×5共25个振动电机,与音频播放同步,使机器人能输出触觉-听觉联合刺激。在32名中国参与者参与的实验中,10种情绪和6种社交手势通过振动、声音或两者结合呈现,参与者在唤醒度和效价量表上进行评分。结果表明:(1)触觉-听觉联合模态显著提升了情绪解码准确率;(2)单个通道(振动或声音)均有效支持特定情绪识别,优势因情绪类型而异;(3)仅靠手势难以清晰传达可区分的情绪。研究凸显了多感官整合在情感化人机交互中的重要性,并揭示了触觉与听觉线索在增强情感交流中的互补作用。

原文摘要 · Abstract (English)

Affective tactile interaction constitutes a fundamental component of human communication. In natural human-human encounters, touch is seldom experienced in isolation; rather, it is inherently multisensory. Individuals not only perceive the physical sensation of touch but also register the accompanying auditory cues generated through contact. The integration of haptic and auditory information forms a rich and nuanced channel for emotional expression. While extensive research has examined how robots convey emotions through facial expressions and speech, their capacity to communicate social gestures and emotions via touch remains largely underexplored. To address this gap, we developed a multimodal interaction system incorporating a 5*5 grid of 25 vibration motors synchronized with audio playback, enabling robots to deliver combined haptic-audio stimuli. In an experiment involving 32 Chinese participants, ten emotions and six social gestures were presented through vibration, sound, or their combination. Participants rated each stimulus on arousal and valence scales. The results revealed that (1) the combined haptic-audio modality significantly enhanced decoding accuracy compared to single modalities; (2) each individual channel-vibration or sound-effectively supported certain emotions recognition, with distinct advantages depending on the emotional expression; and (3) gestures alone were generally insufficient for conveying clearly distinguishable emotions. These findings underscore the importance of multisensory integration in affective human-robot interaction and highlight the complementary roles of haptic and auditory cues in enhancing emotional communication.

情感交互多模态人机协作触觉通信

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。