用扩散模型实现多语言文本精准生成，支持任意语言自由组合。
EasyText: Controllable Diffusion Transformer for Multilingual Text Rendering
- 基于DiT架构，将字符令牌与去噪潜在变量连接，实现多语言文本生成。
- 在100万张合成图像上预训练，2万张高质量图像上微调，效果显著提升。
- 支持位置可控、布局感知的多语言文本渲染，适合跨语言视觉应用。
使用扩散模型生成准确的多语言文本长期以来备受期待但依然困难。近期方法虽在单语言文本渲染上取得进展，但任意语言的渲染仍属空白领域。本文提出EasyText，一种基于扩散Transformer（DiT）的文本渲染框架，将去噪潜在变量与多语言字符令牌相连接。通过引入字符定位编码和位置插值技术，实现可控制且精确的文本渲染。此外，构建了包含100万张多语言图文标注的大型合成数据集，以及2万张高质量标注图像数据集，分别用于预训练与微调。大量实验与评估表明，该方法在多语言文本渲染、视觉质量及布局感知文本融合方面均表现优异。
原文摘要 · Abstract (English)
Generating accurate multilingual text with diffusion models has long been desired but remains challenging. Recent methods have made progress in rendering text in a single language, but rendering arbitrary languages is still an unexplored area. This paper introduces EasyText, a text rendering framework based on DiT (Diffusion Transformer), which connects denoising latents with multilingual character tokens encoded as character tokens. We propose character positioning encoding and position encoding interpolation techniques to achieve controllable and precise text rendering. Additionally, we construct a large-scale synthetic text image dataset with 1 million multilingual image-text annotations as well as a high-quality dataset of 20K annotated images, which are used for pretraining and fine-tuning respectively. Extensive experiments and evaluations demonstrate the effectiveness and advancement of our approach in multilingual text rendering, visual quality, and layout-aware text integration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。