让多角色手绘视频上色更灵活精准，支持自由布局与细节保持
InstanceAnimator: Multi-Instance Sketch Video Colorization
- 通过画布引导条件实现参考元素自由摆放，用户控制更灵活
- 引入实例匹配机制，解决多角色上色错位问题，保证角色一致性
- 自适应解耦控制模块提升细节保真度，适合动画创作与影视预处理
我们提出InstanceAnimator，一种用于多实例手绘视频上色的新型扩散变换框架。现有方法存在三大核心缺陷：过度依赖单参考帧导致用户控制僵化、多角色场景下实例控制差引发错位、细粒度区域细节保真度下降。为此，我们提出三项创新：首先，画布引导条件消除流程碎片化，支持参考元素与背景自由放置，实现前所未有的用户灵活性；其次，实例匹配机制通过将实例特征与草图融合，解决多角色错位问题，确保对多个角色的精确控制；第三，自适应解耦控制模块通过向扩散过程注入角色、背景及文本语义特征，显著提升细节保真度。大量实验表明，InstanceAnimator在多实例上色中实现了更高的视觉质量、更强的实例一致性和更优的用户控制能力。
原文摘要 · Abstract (English)
We propose InstanceAnimator, a novel Diffusion Transformer framework for multi-instance sketch video colorization. Existing methods suffer from three core limitations: inflexible user control due to heavy reliance on single reference frames, poor instance controllability leading to misalignment in multi-character scenarios, and degraded detail fidelity in fine-grained regions. To address these challenges, we introduce three corresponding innovations. First, a Canvas Guidance Condition eliminates workflow fragmentation by allowing free placement of reference elements and background, enabling unprecedented user flexibility. Second, an Instance Matching Mechanism resolves misalignment by integrating instance features with the sketches, ensuring precise control over multiple characters. Third, an Adaptive Decoupled Control Module enhances detail fidelity by injecting semantic features from characters, backgrounds, and text conditions into the diffusion process. Extensive experiments demonstrate that InstanceAnimator achieves superior multi-instance colorization with enhanced user control, high visual quality, and strong instance consistency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。