arXiv:2409.04004cs.CV2024-09ECCV被引 38

仅用一张手写样本,就能生成逼真手写文本。

One-Shot Diffusion Mimicker for Handwritten Text Generation

  • 通过融合高频率特征增强风格提取能力。
  • 单样本生成效果超越需十张样本的旧方法。
  • 适合需要快速个性化手写生成的用户。

现有手写文本生成方法通常需要超过十个手写样本作为风格参考。但在实际应用中,用户更倾向于使用仅需一个参考样本即可运行的模型,以提升便捷性与效率。这种“单样本生成”方式虽简化流程,却因难以从单一样本中准确捕捉作者风格(尤其在字符边缘细节提取时受前景稀疏和背景噪声干扰)而面临挑战。为此,本文提出一种单样本扩散模仿器(One-DM),仅凭一个参考样本即可生成任意书法风格的手写文本。受启发于单一样本的高频信息常蕴含独特风格模式(如字形倾斜度、字母连笔方式),我们设计了一种新颖的风格增强模块,通过引入单一样本的高频成分来提升风格提取精度。随后将风格特征与文本内容融合为联合条件,引导扩散模型生成高质量手写图像。大量实验表明,该方法可在多种语言下仅用一个样本成功生成手写文本,性能甚至优于以往需十余样本的方法。代码已开源:https://github.com/dailenson/One-DM。

原文摘要 · Abstract (English)

Existing handwritten text generation methods often require more than ten handwriting samples as style references. However, in practical applications, users tend to prefer a handwriting generation model that operates with just a single reference sample for its convenience and efficiency. This approach, known as "one-shot generation", significantly simplifies the process but poses a significant challenge due to the difficulty of accurately capturing a writer's style from a single sample, especially when extracting fine details from the characters' edges amidst sparse foreground and undesired background noise. To address this problem, we propose a One-shot Diffusion Mimicker (One-DM) to generate handwritten text that can mimic any calligraphic style with only one reference sample. Inspired by the fact that high-frequency information of the individual sample often contains distinct style patterns (e.g., character slant and letter joining), we develop a novel style-enhanced module to improve the style extraction by incorporating high-frequency components from a single sample. We then fuse the style features with the text content as a merged condition for guiding the diffusion model to produce high-quality handwritten text images. Extensive experiments demonstrate that our method can successfully generate handwriting scripts with just one sample reference in multiple languages, even outperforming previous methods using over ten samples. Our source code is available at https://github.com/dailenson/One-DM.

手写生成扩散模型单样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。