arXiv:2505.03329cs.CV2025-05被引 42

用轻量模块提升多语言文字编辑效果,训练数据仅需0.1百万

FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editing

  • 基于DiT架构,加入轻量图文嵌入模块增强字形理解
  • 仅用0.1M数据训练,比主流方法减少97%数据需求
  • 适合需要低资源、高保真多语言文字编辑的场景

场景文字编辑旨在修改或添加图像中的文字,同时保持文字清晰度和整体视觉质量与背景一致。现有方法主要基于UNet扩散模型,虽有进展,但在复杂字形(尤其是非拉丁文字如中文、韩文、日文)上仍表现不足。为此,我们提出FLUX-Text,一种简单而先进的多语言场景文字编辑扩散Transformer方法。通过轻量级视觉与文本嵌入模块增强字形理解与生成能力,同时保留FLUX原有的生成性能。我们进一步设计了针对文字区域的区域文本感知损失,并采用两阶段训练策略,更好平衡文字编辑与整体图像质量。得益于DiT架构和轻量特征注入模块,FLUX-Text仅需0.1百万训练样本即可训练,相比主流方法所需的2.9百万减少97%。在多个公开数据集(包括英文和中文基准)上的大量实验表明,该方法在视觉质量与文字保真度上均优于现有方法。代码已开源:https://github.com/AMAP-ML/FluxText。

原文摘要 · Abstract (English)

Scene text editing aims to modify or add texts on images while ensuring text fidelity and overall visual quality consistent with the background. Recent methods are primarily built on UNet-based diffusion models, which have improved scene text editing results, but still struggle with complex glyph structures, especially for non-Latin ones (\eg, Chinese, Korean, Japanese). To address these issues, we present \textbf{FLUX-Text}, a simple and advanced multilingual scene text editing DiT method. Specifically, our FLUX-Text enhances glyph understanding and generation through lightweight Visual and Text Embedding Modules, while preserving the original generative capability of FLUX. We further propose a Regional Text Perceptual Loss tailored for text regions, along with a matching two-stage training strategy to better balance text editing and overall image quality. Benefiting from the DiT-based architecture and lightweight feature injection modules, FLUX-Text can be trained with only $0.1$M training examples, a \textbf{97\%} reduction compared to $2.9$M required by popular methods. Extensive experiments on multiple public datasets, including English and Chinese benchmarks, demonstrate that our method surpasses other methods in visual quality and text fidelity. All the code is available at https://github.com/AMAP-ML/FluxText.

文字编辑扩散模型多语言轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。