直接输入视觉信息实现角色动画端到端控制，效果更准。
SCAIL-2: Unifying Controlled Character Animation with End-to-End In-Context Conditioning
- 用原始视觉信息拼接输入，跳过中间表示，减少信息丢失。
- 在MotionPair-60K数据集上达到当前最佳性能，跨任务统一处理。
- 适合做角色动画生成、影视特效或游戏动捕的开发者使用。
可控角色动画旨在将驱动序列中的动作迁移到参考角色上。以往方法严重依赖中间表示（如动作骨骼和遮罩背景），不可避免造成信息损失。本文提出SCAIL-2框架，采用端到端驱动范式，直接将潜空间视觉信息拼接到模型输入序列中。通过构建MotionPair-60K数据集（涵盖多个角色动画子任务）实现端到端训练。利用解耦条件支持不同驱动模式，结合上下文掩码条件与特定模式旋转位置编码（Mode-Specific RoPE），提供超越文本指令和视觉信息的软引导。为缓解细节区域的合成偏差，提出偏差感知的直接偏好优化（Bias-Aware DPO）以构建偏好项。大量实验表明，该方法在多种角色动画任务中均达到领先性能。代码、模型权重及数据集大子集已公开：https://teal024.github.io/SCAIL-2/。
原文摘要 · Abstract (English)
Controlled character animation aims to transfer motion from a driving sequence to a reference character. Prior works heavily rely on intermediate representations, such as pose skeletons for motion and masked backgrounds for environment, inevitably resulting in information loss. In this work, we present SCAIL-2, a framework that adopts an end-to-end driving paradigm by directly concatenating latent visual information to the model's input sequence. We enable end-to-end training through a data synthesis pipeline that produces MotionPair-60K, a curated dataset for several character animation subtasks. We unify the subtasks using decoupled conditions to accommodate different driving patterns, facilitated by In-Context Mask Conditioning and Mode-Specific RoPE, which provide soft guidance beyond textual instructions and visual information. To address synthetic discrepancy in detailed regions, we propose Bias-Aware DPO to construct preference items to mitigate the errors. Extensive experiments demonstrate that our method achieves state-of-the-art performance across various character animation tasks. Code, model weights, and a large subset of the dataset are available at: https://teal024.github.io/SCAIL-2/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。