用Vision Transformer实现单样本手写体生成,风格与内容分离更精准
WriteViT: Handwritten Text Generation with Vision Transformer
- 基于ViT的风格识别器提取书写风格嵌入,结合多尺度Transformer生成器
- 在越语和英语数据集上实现高质量、风格一致的手写文本生成
- 适合低资源场景下的多语言手写体生成与快速风格迁移
人类能仅凭一个样例快速泛化出手写风格,而机器在此类低数据场景下常忽略细微的空间与风格特征。为此,我们提出WriteViT,一种基于Vision Transformer(ViT)的一次性手写文本合成框架。该框架包含基于ViT的书写者识别器以提取风格嵌入,采用带条件位置编码(CPE)的Transformer编码器-解码器块构建的多尺度生成器,以及轻量级ViT识别器。相比以往依赖CNN或CRNN的方法,本设计通过关键组件引入Transformer,更有效地捕捉笔画细节与高层风格信息。尽管手写文本合成已广泛研究,其在越南语(富含变音符号与复杂排版)中的应用仍有限。在越南语与英语数据集上的实验表明,WriteViT在低资源条件下仍能生成高质量、风格一致的手写文本,并保持良好识别性能。结果凸显了基于Transformer的设计在多语言手写生成与高效风格适配中的潜力。
原文摘要 · Abstract (English)
Humans can quickly generalize handwriting styles from a single example by intuitively separating content from style. Machines, however, struggle with this task, especially in low-data settings, often missing subtle spatial and stylistic cues. Motivated by this gap, we introduce WriteViT, a one-shot handwritten text synthesis framework that incorporates Vision Transformers (ViT), a family of models that have shown strong performance across various computer vision tasks. WriteViT integrates a ViT-based Writer Identifier for extracting style embeddings, a multi-scale generator built with Transformer encoder-decoder blocks enhanced by conditional positional encoding (CPE), and a lightweight ViT-based recognizer. While previous methods typically rely on CNNs or CRNNs, our design leverages transformers in key components to better capture both fine-grained stroke details and higher-level style information. Although handwritten text synthesis has been widely explored, its application to Vietnamese -- a language rich in diacritics and complex typography -- remains limited. Experiments on Vietnamese and English datasets demonstrate that WriteViT produces high-quality, style-consistent handwriting while maintaining strong recognition performance in low-resource scenarios. These results highlight the promise of transformer-based designs for multilingual handwriting generation and efficient style adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。