用扩散模型生成连贯手写文本行，兼顾风格与内容准确
Beyond Isolated Words: Diffusion Brush for Handwritten Text-Line Generation
- 分离内容与风格学习，通过行列掩码捕捉字内字间样式
- 引入多尺度内容判别器，确保整体连贯与局部精准
- 适合手写体生成、古籍数字化等需真实排版的场景
现有手写文本生成方法主要关注孤立单词。但真实手写文本不仅需关注单个词，还需考虑词间关系，如垂直对齐和水平间距。因此，生成完整文本行成为更全面且有前景的任务。然而该任务面临巨大挑战：需准确建模包含字内与字间关系的复杂风格模式，并在大量字符中保持内容准确性。为此，我们提出 DiffBrush，一种基于扩散模型的手写文本行生成新方法。不同于现有方法，DiffBrush 通过两项关键策略同时提升风格模仿与内容准确性：(1) 内容解耦风格学习，利用列和行掩码将风格与内容解离，更好捕捉字内与字间风格模式；(2) 多尺度内容学习，采用行级与词级判别器，确保文本内容的全局连贯性与局部准确性。大量实验表明，DiffBrush 在生成高质量文本行方面表现优异，尤其在风格再现与内容保留上优势显著。代码已开源：https://github.com/dailenson/DiffBrush。
原文摘要 · Abstract (English)
Existing handwritten text generation methods primarily focus on isolated words. However, realistic handwritten text demands attention not only to individual words but also to the relationships between them, such as vertical alignment and horizontal spacing. Therefore, generating entire text lines emerges as a more promising and comprehensive task. However, this task poses significant challenges, including the accurate modeling of complex style patterns encompassing both intra- and inter-word relationships, and maintaining content accuracy across numerous characters. To address these challenges, we propose DiffBrush, a novel diffusion-based model for handwritten text-line generation. Unlike existing methods, DiffBrush excels in both style imitation and content accuracy through two key strategies: (1) content-decoupled style learning, which disentangles style from content to better capture intra-word and inter-word style patterns by using column- and row-wise masking; and (2) multi-scale content learning, which employs line and word discriminators to ensure global coherence and local accuracy of textual content. Extensive experiments show that DiffBrush excels in generating high-quality text lines, particularly in style reproduction and content preservation. Code is available at https://github.com/dailenson/DiffBrush.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。