arXiv:2606.16600cs.RO2026-06

让机器人说话时手势更自然,精准同步且不越界。

WaveSync: Constrained Wavefront Optimization for Synchronized Co-Speech Gestures in Humanoid Robots

论文配图:WaveSync: Constrained Wavefront Optimization for Synchronized Co-Speech Gestures in Humanoid Robots
图 1 · 摘自论文原文
  • 用大模型分析语义重要性,生成连续波形指导手势
  • 优化后手势与语音峰值对齐,误差小于0.1秒
  • 适合需要真实交互的机器人研发者使用

自然的人机交互离不开富有表现力的伴随手势,但物理类人机器人因运动学与动力学约束,难以实现手势与语音强调的精确同步。与虚拟形象不同,类人机器人无法自由执行快速或重叠动作,导致词级同步与硬件安全运动规划成为耦合难题。本文提出WaveSync,一种混合框架:利用大语言模型将对话回复分解为结构化语义模板,并为每个词分配重要性权重,构建连续的语义重要性波;通过动态运动基元生成轨迹,在保证运动可行性的同时增强表现力;再通过波前优化阶段实现峰峰同步,并采用手势持续时间压缩与前向传播修复残余运动学冲突。基于五个对话场景的实验表明,该方法在客观与主观评估中均优于三种基线,同步精度高,且各组件协同保障手势的表达性、语义相关性与运动合规性。代码、资源及演示视频见https://github.com/pairs-lab/WaveSync。

原文摘要 · Abstract (English)

Expressive co-speech gestures are crucial for natural human-robot interaction, but generating them on physical humanoid robots is difficult because gesture strokes must align with speech emphasis while satisfying strict kinematic and dynamic constraints. Unlike virtual avatars, humanoid robots cannot freely execute rapid or overlapping motions, making word-level synchronization and hardware-safe motion planning a coupled problem. We present \textbf{WaveSync}, a hybrid framework in which a Large Language Model decomposes dialogue responses into structured semantic schemas and assigns per-word importance weights, constructing a continuous Semantic Importance Wave. Gesture trajectories are shaped through Dynamic Movement Primitives, enforcing kinematic feasibility while enhancing expressiveness. A Wavefront Optimization stage aligns peak-to-peak gesture-speech synchronization and resolves residual kinematic violations through gesture-duration compression and forward propagation. Experimental evaluation based on five dialogue scenarios shows that our method achieves high synchronization accuracy and outperforms three baselines in both objective and subjective evaluations. Each component in WaveSync plays a necessary role in producing gestures that are expressive, semantically grounded, and kinematically compliant. The code, resources, and videos are available at \href{https://github.com/pairs-lab/WaveSync}{WaveSync}

类人机器人手势生成语音同步运动规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。