arXiv:2609.04545cs.ROcs.CV2026-09

让机器人实时识别社交手势,适应遮挡和延迟挑战。

SocioGesture: Real-Time and Adaptive Social Gesture Perception for Human-Robot Interaction

论文配图:SocioGesture: Real-Time and Adaptive Social Gesture Perception for Human-Robot Interaction
图 1 · 摘自论文原文
  • 用轻量双流模型融合身体动作与手部姿态,实现低延迟识别。
  • 在遮挡条件下仍保持高准确率,边缘设备上实时运行。
  • 支持在线学习新手势,适合真实人机交互场景。

在真实人机交互部署中,机器人需从噪声感知数据中识别邀请、拒绝等社交手势,面临部分遮挡、视角变化和严格延迟限制。本文提出SocioGesture系统,采用紧凑的置信度感知躯干-手部骨骼表示,结合轻量级双流模型,融合身体运动与手部动作实现低延迟本地识别。为提升鲁棒性,训练时引入遮挡感知的骨骼污染机制,模拟缺失手部、遮挡手臂及关键点不稳定,且不增加推理开销。在混合室内外场景收集的社交手势数据集上,SocioGesture实现强跨主体识别性能,在结构化关节遮挡下显著提升鲁棒性,并可在机器人搭载的边缘设备上实时运行。部署中,不确定交互段落被保存用于离线标注与适应,使系统可扩展手势词表,同时保持原有类别性能。结果表明,该系统为交互机器人提供了高效、鲁棒且可自适应的社会感知路径。

原文摘要 · Abstract (English)

Robots interacting with people must recognize not only explicit commands, but also social cues such as invitations, refusals, and unavailability. In real deployments, these cues must be inferred from noisy onboard perception under partial occlusion, changing viewpoints, and strict latency constraints. We present SocioGesture, a real-time adaptive social gesture perception system for human-robot interaction (HRI). SocioGesture uses a compact confidence-aware body-hand skeleton representation and a lightweight dual-stream model that fuses body motion with hand articulation for low-latency onboard recognition. To improve deployment robustness, we train the model with occlusion-aware skeleton corruption, exposing it to missing hands, occluded arms, and temporally unstable keypoints without increasing the inference cost. On a social gesture dataset collected in mixed indoor-outdoor HRI scenarios, SocioGesture achieves strong held-out-subject recognition, substantially improves robustness under structured joint occlusion, and runs in real time on a robot-mounted edge device. During deployment, uncertain interaction segments are saved for offline labeling and adaptation, enabling SocioGesture to expand its gesture vocabulary while preserving performance in the original classes. These results demonstrate a practical path toward robust, efficient, and adaptive social perception for interactive robots.

人机交互手势识别边缘计算自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。