支持多参考帧的灵活视频着色，提升颜色一致性与稳定性。
TimeColor: Flexible Reference Colorization via Temporal Concatenation
- 将多个参考图像作为时序潜变量拼接输入，统一处理不同来源参考。
- 在Sakuga-42M上相比基线提升颜色保真度、身份一致性和时间稳定性。
- 适合需要多参考控制的动画着色场景，如角色设定或背景参考。
大多数着色模型仅依赖单一参考（通常是场景首帧），忽略了角色设定图、背景图或任意着色帧等其他条件数据。我们提出TimeColor，一种基于草图的视频着色模型，支持异构、数量可变的多参考输入，并通过显式的每参考区域分配实现精准绑定。TimeColor将参考编码为额外的潜变量帧，进行时序拼接，使它们在每个扩散步骤中可并行处理，同时保持模型参数量不变。此外，采用时空对应掩码注意力机制强化主体-参考绑定，并结合模态解耦的RoPE索引，缓解捷径学习和跨身份调色板泄露问题。在Sakuga-42M数据集上，无论是单参考还是多参考设置下，TimeColor均显著优于先前基线，在颜色保真度、身份一致性与时间稳定性方面表现更优。
原文摘要 · Abstract (English)
Most colorization models condition only on a single reference, typically the first frame of the scene. However, this approach ignores other sources of conditional data, such as character sheets, background images, or arbitrary colorized frames. We propose TimeColor, a sketch-based video colorization model that supports heterogeneous, variable-count references with the use of explicit per-reference region assignment. TimeColor encodes references as additional latent frames which are concatenated temporally, permitting them to be processed concurrently in each diffusion step while keeping the model's parameter count fixed. TimeColor also uses spatiotemporal correspondence-masked attention to enforce subject -- reference binding in addition to modality-disjoint RoPE indexing. These mechanisms mitigate shortcutting and cross-identity palette leakage. Experiments on Sakuga-42M under both single- and multi-reference protocols show that TimeColor improves color fidelity, identity consistency, and temporal stability over prior baselines. Our project page is available at https://bconstantine.github.io/TimeColor/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。