让AI精准生成和编辑设计图中的中英文字体,保持风格一致。
UTDesign: A Unified Framework for Stylized Text Editing and Generation in Graphic Design Images
- 基于DiT架构训练新文本风格迁移模型,生成带透明背景的字体。
- 支持根据背景、提示和布局生成准确且风格统一的文字。
- 适配海报广告等设计场景,开源代码可复现效果。
AI辅助图形设计已成为自动化创建和编辑海报、横幅及广告等设计元素的强大工具。尽管基于扩散模型的文生图方法在视觉内容生成方面表现优异,但在小字号字体和非拉丁文字(如中文)渲染上仍存在局限。本文提出UTDesign,一个统一框架,用于设计图像中风格化文字的高精度编辑与条件生成,支持英汉双语。该框架引入一种从零训练的基于DiT的文本风格迁移模型,在合成数据集上训练,可生成保留参考字形风格的透明RGBA文字前景。进一步通过在带详细文本标注的精选数据集上训练多模态条件编码器,将该模型扩展为条件文本生成框架,实现基于背景图、提示词和布局规格的精确、风格一致的文字合成。最后,结合预训练文生图模型与基于MLLM的布局规划器,构建全自动文生设计(T2D)流水线。大量实验表明,UTDesign在开源方法中达到当前最佳水平,在风格一致性与文字准确性方面均优于现有方案,且相较商业专有系统具独特优势。代码与数据已公开于https://github.com/ZYM-PKU/UTDesign。
原文摘要 · Abstract (English)
AI-assisted graphic design has emerged as a powerful tool for automating the creation and editing of design elements such as posters, banners, and advertisements. While diffusion-based text-to-image models have demonstrated strong capabilities in visual content generation, their text rendering performance, particularly for small-scale typography and non-Latin scripts, remains limited. In this paper, we propose UTDesign, a unified framework for high-precision stylized text editing and conditional text generation in design images, supporting both English and Chinese scripts. Our framework introduces a novel DiT-based text style transfer model trained from scratch on a synthetic dataset, capable of generating transparent RGBA text foregrounds that preserve the style of reference glyphs. We further extend this model into a conditional text generation framework by training a multi-modal condition encoder on a curated dataset with detailed text annotations, enabling accurate, style-consistent text synthesis conditioned on background images, prompts, and layout specifications. Finally, we integrate our approach into a fully automated text-to-design (T2D) pipeline by incorporating pre-trained text-to-image (T2I) models and an MLLM-based layout planner. Extensive experiments demonstrate that UTDesign achieves state-of-the-art performance among open-source methods in terms of stylistic consistency and text accuracy, and also exhibits unique advantages compared to proprietary commercial approaches. Code and data for this paper are available at https://github.com/ZYM-PKU/UTDesign.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。