用可编程符号脚本精准控制动作方向与节奏,让文本生成动作更自然可控。
SOSControl: Enhancing Human Motion Generation through Saliency-Aware Symbolic Orientation and Timing Control

- 提出SOS符号脚本,用关键帧标注身体部位朝向与时间节奏。
- 自动提取脚本可生成稀疏、可读的符号化动作描述,准确率超基准方法。
- 适合需要精细动作控制的研究者,尤其在舞蹈/动画生成场景中表现优异。
传统文本到动作生成框架缺乏精确控制,现有基于关节关键帧位置的方法仅提供位置引导,难以直观指定身体部位朝向和运动节奏。为此,我们提出显著朝向符号(SOS)脚本,一种可编程的符号化框架,用于在关键帧上指定身体部位朝向与运动时间。我们进一步设计了自动SOS提取管道,采用时序约束的凝聚聚类检测帧显著性,并通过基于显著性的掩码方案(SMS)从运动数据中直接生成稀疏、可解释的SOS脚本。此外,我们提出SOSControl框架,将稀疏SOS脚本中的朝向符号视为显著约束,在生成过程中优先满足这些条件。通过SMS数据增强与基于梯度的迭代优化,框架显著提升对用户指定约束的对齐能力。同时采用基于ControlNet的ACTOR-PAE解码器,确保输出动作流畅自然。大量实验表明,SOS提取管道能生成人类可读的符号化脚本,而SOSControl在动作质量、可控性与泛化能力方面均优于现有基线,尤其在运动节奏与身体部位朝向控制上表现突出。
原文摘要 · Abstract (English)
Traditional text-to-motion frameworks often lack precise control, and existing approaches based on joint keyframe locations provide only positional guidance, making it challenging and unintuitive to specify body part orientations and motion timing. To address these limitations, we introduce the Salient Orientation Symbolic (SOS) script, a programmable symbolic framework for specifying body part orientations and motion timing at keyframes. We further propose an automatic SOS extraction pipeline that employs temporally-constrained agglomerative clustering for frame saliency detection and a Saliency-based Masking Scheme (SMS) to generate sparse, interpretable SOS scripts directly from motion data. Moreover, we present the SOSControl framework, which treats the available orientation symbols in the sparse SOS script as salient and prioritizes satisfying these constraints during motion generation. By incorporating SMS-based data augmentation and gradient-based iterative optimization, the framework enhances alignment with user-specified constraints. Additionally, it employs a ControlNet-based ACTOR-PAE Decoder to ensure smooth and natural motion outputs. Extensive experiments demonstrate that the SOS extraction pipeline generates human-interpretable scripts with symbolic annotations at salient keyframes, while the SOSControl framework outperforms existing baselines in motion quality, controllability, and generalizability with respect to motion timing and body part orientation control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。