让机器人通过虚拟现实学习自然表情互动，实时反应更流畅。
FABG : End-to-end Imitation Learning for Embodied Affective Human-Robot Interaction
- 用虚拟现实让操作者直接代入机器人视角，实时生成行为示范
- 提出预测式延迟补偿，显著降低机器人响应延迟，提升交互流畅度
- 无需手动编写动作，可自动学习人类直觉性交互行为，适合人机共融场景
本文提出FABG（面部情感行为生成）系统，一种面向具身情感人机交互的端到端模仿学习方法，旨在生成自然流畅的面部情感行为。在交互中，获取高质量示范仍是一大挑战。为此，我们开发了一套沉浸式虚拟现实（VR）示范系统，使操作者能感知立体环境，确保“操作者的视觉感知与机器人的感官输入一致”且“操作者的行为直接决定机器人的动作”，仿佛操作者替代机器人参与人际互动。我们提出一种预测驱动的延迟补偿策略，以减少机器人反应延迟，提升交互流畅性。FABG能够自然捕捉人类交互行为及由直觉驱动的潜意识动作，避免了手动行为脚本编写。我们在一台25自由度（DoF）的真实人形机器人上部署FABG，通过四个基础交互任务——表情响应、动态凝视、中心视野注意力和手势识别——验证其有效性，涵盖数据采集与策略训练全过程。
原文摘要 · Abstract (English)
This paper proposes FABG (Facial Affective Behavior Generation), an end-to-end imitation learning system for human-robot interaction, designed to generate natural and fluid facial affective behaviors. In interaction, effectively obtaining high-quality demonstrations remains a challenge. In this work, we develop an immersive virtual reality (VR) demonstration system that allows operators to perceive stereoscopic environments. This system ensures "the operator's visual perception matches the robot's sensory input" and "the operator's actions directly determine the robot's behaviors" - as if the operator replaces the robot in human interaction engagements. We propose a prediction-driven latency compensation strategy to reduce robotic reaction delays and enhance interaction fluency. FABG naturally acquires human interactive behaviors and subconscious motions driven by intuition, eliminating manual behavior scripting. We deploy FABG on a real-world 25-degree-of-freedom (DoF) humanoid robot, validating its effectiveness through four fundamental interaction tasks: expression response, dynamic gaze, foveated attention, and gesture recognition, supported by data collection and policy training. Project website: https://cybergenies.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。