arXiv:2507.00992cs.CV2025-07ICCV被引 8

用像素级文字掩码实现精准文字生成,支持中英文混合与复杂排版。

UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesis

  • 以文字掩码为统一条件,结合分割模型与扩散模型生成高保真文本。
  • 在AnyText、GlyphMM和MiniText三个基准上均超越现有方法,小字生成提升显著。
  • 适合需要精确文字渲染的场景,如广告设计、UI生成和多语言内容创作。

文本到图像生成虽已大幅推动内容创作,但精准渲染视觉文字仍面临字符模糊、语义漂移和风格控制不足等挑战。现有方法常依赖预渲染的文字图像作为条件,难以保留原始字体样式与色彩信息,需采用复杂的多分支结构,增加模型开销且降低灵活性。为此,我们提出一种基于分割引导的统一框架,使用富含字形、颜色与空间细节的像素级文字掩码作为统一条件输入。该方法包含两个核心组件:(1) 经微调的双语分割模型,用于精准提取文字掩码;(2) 增强的轻量级扩散模型,引入自适应字形条件与区域特定损失,以保持内容与风格的一致性。在AnyText基准上达到当前最优性能,显著优于以往方法,涵盖中英文场景。为实现更严格的评估,我们还构建了两个新基准:GlyphMM-benchmark用于测试复杂排版中的布局与字形一致性,MiniText-benchmark用于评估小尺度文本区域的生成质量。实验表明,本模型在两类场景下均大幅领先,尤其在小文本渲染与复杂布局保持方面表现突出,验证了其强大的泛化能力与部署潜力。

原文摘要 · Abstract (English)

Text-to-image generation has greatly advanced content creation, yet accurately rendering visual text remains a key challenge due to blurred glyphs, semantic drift, and limited style control. Existing methods often rely on pre-rendered glyph images as conditions, but these struggle to retain original font styles and color cues, necessitating complex multi-branch designs that increase model overhead and reduce flexibility. To address these issues, we propose a segmentation-guided framework that uses pixel-level visual text masks -- rich in glyph shape, color, and spatial detail -- as unified conditional inputs. Our method introduces two core components: (1) a fine-tuned bilingual segmentation model for precise text mask extraction, and (2) a streamlined diffusion model augmented with adaptive glyph conditioning and a region-specific loss to preserve textual fidelity in both content and style. Our approach achieves state-of-the-art performance on the AnyText benchmark, significantly surpassing prior methods in both Chinese and English settings. To enable more rigorous evaluation, we also introduce two new benchmarks: GlyphMM-benchmark for testing layout and glyph consistency in complex typesetting, and MiniText-benchmark for assessing generation quality in small-scale text regions. Experimental results show that our model outperforms existing methods by a large margin in both scenarios, particularly excelling at small text rendering and complex layout preservation, validating its strong generalization and deployment readiness.

文本生成扩散模型图像合成文字渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。