无需真人绘画数据，自监督生成逼真作画过程视频
AnimatePainter: A Self-Supervised Rendering Framework for Reconstructing Painting Process
- 将作画过程视为逆向去笔触的视频生成任务
- 通过深度估计和笔触渲染构建自监督数据集
- 适合艺术生成、数字绘画研究者使用
人类能直观地将图像分解为一系列笔触来创作绘画，但现有生成作画过程的方法受限于特定数据类型，且常依赖昂贵的人工标注数据集。我们提出一种新型自监督框架，可从任意图像生成作画过程，将该任务视为视频生成问题。方法通过逐步从参考图像中移除笔触，模拟类人创作序列。关键在于，本方法无需真实人类作画过程数据；而是利用深度估计与笔触渲染构建自监督数据集。我们将人类绘画建模为‘精修’与‘分层’过程，并引入深度融合层，使视频生成模型能够学习并复现人类作画行为。大量实验验证了方法的有效性，证明其可在无需真实作画数据的情况下生成逼真的绘画过程。
原文摘要 · Abstract (English)
Humans can intuitively decompose an image into a sequence of strokes to create a painting, yet existing methods for generating drawing processes are limited to specific data types and often rely on expensive human-annotated datasets. We propose a novel self-supervised framework for generating drawing processes from any type of image, treating the task as a video generation problem. Our approach reverses the drawing process by progressively removing strokes from a reference image, simulating a human-like creation sequence. Crucially, our method does not require costly datasets of real human drawing processes; instead, we leverage depth estimation and stroke rendering to construct a self-supervised dataset. We model human drawings as "refinement" and "layering" processes and introduce depth fusion layers to enable video generation models to learn and replicate human drawing behavior. Extensive experiments validate the effectiveness of our approach, demonstrating its ability to generate realistic drawings without the need for real drawing process data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。