让动作生成更精准：用骨骼信息增强文本驱动动作建模
SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editing
- 显式建模关节、帧与文本词间的复杂关系
- 零样本编辑仅靠文本提示即可实现动作修改
- 无需微调，适合快速动作生成与编辑应用
文本驱动的动作生成随着去噪扩散模型的发展取得显著进展。然而，以往方法常对骨骼关节、时间帧和文本词进行过度简化，限制了各模态内部及相互间信息的充分捕捉。此外，使用预训练模型进行下游任务（如编辑）时，通常需要额外操作，包括手动干预、优化或微调。本文提出骨架感知的潜在扩散模型（SALAD），显式捕捉关节、帧与词之间的复杂关联。通过利用生成过程中产生的跨注意力图，我们实现了基于注意力的零样本文本驱动动作编辑，仅需文本提示，无需额外用户输入。该方法在文本-动作对齐上显著优于先前方法，且不牺牲生成质量，展现出多样化的编辑能力。代码可在项目页面获取。
原文摘要 · Abstract (English)
Text-driven motion generation has advanced significantly with the rise of denoising diffusion models. However, previous methods often oversimplify representations for the skeletal joints, temporal frames, and textual words, limiting their ability to fully capture the information within each modality and their interactions. Moreover, when using pre-trained models for downstream tasks, such as editing, they typically require additional efforts, including manual interventions, optimization, or fine-tuning. In this paper, we introduce a skeleton-aware latent diffusion (SALAD), a model that explicitly captures the intricate inter-relationships between joints, frames, and words. Furthermore, by leveraging cross-attention maps produced during the generation process, we enable attention-based zero-shot text-driven motion editing using a pre-trained SALAD model, requiring no additional user input beyond text prompts. Our approach significantly outperforms previous methods in terms of text-motion alignment without compromising generation quality, and demonstrates practical versatility by providing diverse editing capabilities beyond generation. Code is available at project page.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。