无需标注，自动预测漫画线稿区域对应关系
Region-Wise Correspondence Prediction between Manga Line Art Images
- 基于Transformer框架，利用自动生成数据训练
- 实现78.4%-84.4%的区域级对应准确率
- 适合漫画上色与动画帧生成等下游任务
理解漫画线稿图像间的区域级对应关系是高级漫画处理的基础，支持着上色、中间帧生成等下游任务。与富含视觉线索的自然图像不同，漫画线稿仅由稀疏的黑白线条构成，难以判断跨图像的区域对应关系。本文提出一项新任务：在无任何标注的情况下，预测原始漫画线稿图像间的区域级对应关系。为此，我们设计了一种基于Transformer的框架，利用大规模自动生成的区域对应关系进行训练，使模型学会抑制噪声匹配、强化一致的结构关系，从而实现图像内与跨图像的鲁棒块级特征对齐。推理时，方法通过边缘感知聚类和区域匹配，分割线稿并建立连贯的区域级对应关系。我们构建了人工标注的基准测试集，多数据集实验表明，该方法在块级精度和区域级对应性能上均表现优异，区域级准确率达到78.4%-84.4%。结果验证了该方法在真实漫画与动画应用中的潜力。
原文摘要 · Abstract (English)
Understanding region-wise correspondences between manga line art images is fundamental for high-level manga processing, supporting downstream tasks such as line art colorization and in-between frame generation. Unlike natural images that contain rich visual cues, manga line art consists only of sparse black-and-white strokes, making it challenging to determine which regions correspond across images. In this work, we introduce a new task: predicting region-wise correspondence between raw manga line art images without any annotations. To address this problem, we propose a Transformer-based framework trained on large-scale, automatically generated region correspondences. The model learns to suppress noisy matches and strengthen consistent structural relationships, resulting in robust patch-level feature alignment within and across images. During inference, our method segments each line art and establishes coherent region-level correspondences through edge-aware clustering and region matching. We construct manually annotated benchmarks for evaluation, and experiments across multiple datasets demonstrate both high patch-level accuracy and strong region-level correspondence performance, achieving 78.4-84.4% region-level accuracy. These results highlight the potential of our method for real-world manga and animation applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。