无需特定角色数据，任意角色动作都能流畅生成中间帧。
AnyMoLe: Any Character Motion In-betweening Leveraging Video Diffusion Models
- 用视频扩散模型分两阶段生成帧，增强上下文理解。
- 通过微调技术实现真实与渲染动画的无缝衔接。
- 支持任意骨骼结构角色，适合通用动作补全任务。
尽管基于学习的动作补全近期取得进展，但一个关键局限被忽视：需依赖特定角色的数据集。本文提出AnyMoLe，利用视频扩散模型在无外部数据条件下为任意角色生成中间动作帧。方法采用两阶段帧生成流程以提升上下文理解能力，并引入ICAdapt微调技术，弥合真实世界与渲染角色动画之间的域差距。此外，提出“动作-视频模仿”优化策略，结合2D与3D感知特征，实现对任意关节结构角色的自然动作生成。AnyMoLe显著降低数据依赖性，生成平滑逼真的过渡动作，适用于广泛的动作补全任务。
原文摘要 · Abstract (English)
Despite recent advancements in learning-based motion in-betweening, a key limitation has been overlooked: the requirement for character-specific datasets. In this work, we introduce AnyMoLe, a novel method that addresses this limitation by leveraging video diffusion models to generate motion in-between frames for arbitrary characters without external data. Our approach employs a two-stage frame generation process to enhance contextual understanding. Furthermore, to bridge the domain gap between real-world and rendered character animations, we introduce ICAdapt, a fine-tuning technique for video diffusion models. Additionally, we propose a ``motion-video mimicking'' optimization technique, enabling seamless motion generation for characters with arbitrary joint structures using 2D and 3D-aware features. AnyMoLe significantly reduces data dependency while generating smooth and realistic transitions, making it applicable to a wide range of motion in-betweening tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。