arXiv:2603.15975cs.CV2026-03被引 1

一个统一框架让动作大模型支持多种新任务,无需重新训练。

UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priors

  • 将不同任务拆解为帧级操作组合,实现上下文自适应。
  • 在多个任务上超越专用模型和零样本基线,效果稳定提升。
  • 适合想快速扩展动作生成能力的研究者与开发者使用。

大规模基础模型(LFMs)通过海量3D人体动作数据和对应文本描述,已在文本到动作生成任务中取得显著进展,学习到强大的生成先验。然而,如何高效利用这类单一用途的动作大模型(如仅支持文本到动作合成)来应对更广泛的跨模态、上下文感知的下游动作生成任务仍不明确。以往工作通常对每个下游任务进行特定适配。本文提出UMO,一种简单而通用的统一框架,将多样下游任务统一为原子帧级操作的组合,实现对预训练DiT架构动作大模型生成先验的上下文解锁。UMO引入三个可学习的帧级元操作嵌入以指定每帧意图,并采用轻量级时序融合机制注入上下文线索,相比基础模型运行开销极低。该设计使原本仅支持文本到动作生成的模型,经微调后可支持多种此前不支持的任务,包括时间插补、文本引导动作编辑、文本序列化几何约束及多身份反应生成。实验表明,尽管仅用单一统一模型,UMO在广泛基准上持续优于任务专用与无训练基线。

原文摘要 · Abstract (English)

Large-scale foundation models (LFMs) have recently made impressive progress in text-to-motion generation by learning strong generative priors from massive 3D human motion datasets and paired text descriptions. However, how to effectively and efficiently leverage such single-purpose motion LFMs, i.e., text-to-motion synthesis, in more diverse cross-modal and in-context motion generation downstream tasks remains largely unclear. Prior work typically adapts pretrained generative priors to individual downstream tasks in a task-specific manner. In contrast, our goal is to unlock such priors to support a broad spectrum of downstream motion generation tasks within a single unified framework. To bridge this gap, we present UMO, a simple yet general unified formulation that casts diverse downstream tasks into compositions of atomic per-frame operations, enabling in-context adaptation to unlock the generative priors of pretrained DiT-based motion LFMs. Specifically, UMO introduces three learnable frame-level meta-operation embeddings to specify per-frame intent and employs lightweight temporal fusion to inject in-context cues into the pretrained backbone, with negligible runtime overhead compared to the base model. With this design, UMO finetunes the pretrained model, originally limited to text-to-motion generation, to support diverse previously unsupported tasks, including temporal inpainting, text-guided motion editing, text-serialized geometric constraints, and multi-identity reaction generation. Experiments demonstrate that UMO consistently outperforms task-specific and training-free baselines across a wide range of benchmarks, despite using a single unified model. Code and model will be publicly available. Project Page: https://oliver-cong02.github.io/UMO.github.io/

动作生成上下文学习统一框架扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。