用单步预测加速绘画生成,提升质量与效率。
AttentionPainter: An Efficient and Adaptive Stroke Predictor for Scene Painting
- 单次前向传播预测大量笔触参数,取代迭代生成。
- 训练速度提升13倍,重建质量更高。
- 适合需要快速创作与编辑的艺术家使用。
基于笔触的渲染(SBR)旨在将输入图像分解为一系列参数化笔触,并将其重绘为接近原图的绘画作品。近年来,基于深度学习和强化学习的神经绘画方法被提出,但存在推理时间长或训练不稳定的缺陷。为此,我们提出AttentionPainter,一种高效且自适应的单步神经绘画模型。首先,设计了一种可扩展的笔触预测器,在单次前向传播中预测大量笔触参数,避免了以往强化学习或自回归方法的逐次预测,显著提升速度。为提高训练效率,提出Fast Stroke Stacking算法,使训练加速13倍。此外,引入Stroke-density Loss,促使模型在细节区域使用更小的笔触,从而提升重建质量。最后,提出一种新的笔触扩散模型,可在笔触参数空间中去噪,支持条件与无条件的笔触生成,便于实现笔触修复与编辑,助力艺术家创作。大量实验表明,AttentionPainter优于现有最先进神经绘画方法。
原文摘要 · Abstract (English)
Stroke-based Rendering (SBR) aims to decompose an input image into a sequence of parameterized strokes, which can be rendered into a painting that resembles the input image. Recently, Neural Painting methods that utilize deep learning and reinforcement learning models to predict the stroke sequences have been developed, but suffer from longer inference time or unstable training. To address these issues, we propose AttentionPainter, an efficient and adaptive model for single-step neural painting. First, we propose a novel scalable stroke predictor, which predicts a large number of stroke parameters within a single forward process, instead of the iterative prediction of previous Reinforcement Learning or auto-regressive methods, which makes AttentionPainter faster than previous neural painting methods. To further increase the training efficiency, we propose a Fast Stroke Stacking algorithm, which brings 13 times acceleration for training. Moreover, we propose Stroke-density Loss, which encourages the model to use small strokes for detailed information, to help improve the reconstruction quality. Finally, we propose a new stroke diffusion model for both conditional and unconditional stroke-based generation, which denoises in the stroke parameter space and facilitates stroke-based inpainting and editing applications helpful for human artists design. Extensive experiments show that AttentionPainter outperforms the state-of-the-art neural painting methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。