用大模型统一处理动作理解、生成与编辑，支持单人多人场景。
MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing
- 用残差量化将动作序列转为多流离散标记，提升表示效率。
- 延迟并行建模让大模型高效捕捉多流动作依赖，计算开销接近单流。
- 双塔架构分离语言与动作模态参数，避免信息干扰，提升任务稳定性。
本文提出 MotionVerse，一个统一框架，利用大语言模型（LLMs）实现单人和多人场景下的动作理解、生成与编辑。为高效表示动作数据，我们采用残差量化运动分词器，将连续动作序列转化为多流离散标记。此外，提出 extit{延迟并行} 建模策略，对残差标记流进行时间错位编码，使 LLM 能有效捕获跨流依赖，同时保持与单流建模相当的计算效率。为缓解动作与语言间的模态干扰，设计了具有模态专用参数的 extit{双塔架构}，确保动作信息在理解与生成任务中稳定融合。全面消融实验验证了各组件有效性，大量实验展示其在多种动作相关任务上的优越性能。
原文摘要 · Abstract (English)
This paper proposes MotionVerse, a unified framework that harnesses the capabilities of Large Language Models (LLMs) to comprehend, generate, and edit human motion in both single-person and multi-person scenarios. To efficiently represent motion data, we employ a motion tokenizer with residual quantization, which converts continuous motion sequences into multi-stream discrete tokens. Furthermore, we introduce a \textit{Delay Parallel} Modeling strategy, which temporally staggers the encoding of residual token streams. This design enables LLMs to effectively capture inter-stream dependencies while maintaining computational efficiency comparable to single-stream modeling. Moreover, to alleviate modality interference between motion and language, we design a \textit{dual-tower architecture} with modality-specific parameters, ensuring stable integration of motion information for both comprehension and generation tasks. Comprehensive ablation studies demonstrate the effectiveness of each component in MotionVerse, and extensive experiments showcase its superior performance across a wide range of motion-relevant tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。