用统一模型同时提升文本图像分辨率和文字清晰度。
DualTSR: Unified Dual-Diffusion Transformer for Scene Text Image Super-Resolution
- 双扩散机制让模型自研文本先验,无需外接OCR。
- 在合成中文数据集上达到领先保真度,真实场景也表现优秀。
- 结构简洁,端到端训练,适合需要高精度文本恢复的场景。
场景文本图像超分辨率(STISR)旨在恢复低分辨率文本图像中的高分辨率细节,这对人眼可读性和机器识别都至关重要。现有方法通常依赖外部光学字符识别(OCR)模型提供文本先验,或采用复杂多组件架构,难以训练和复现。本文提出DualTSR,一种统一的端到端框架,通过单一多模态变压器骨干网络,结合双扩散目标进行训练:一方面利用条件流匹配建模高分辨率图像的连续分布,另一方面通过离散扩散建模文本内容的离散分布。该共享设计使视觉与文本信息在每一层交互,使模型能够内生推断文本先验,无需依赖外部OCR模块。相比以往多分支扩散系统,DualTSR具有更简洁的端到端结构,手工作坊组件更少。在合成中文基准数据集和精心设计的真实世界评估协议上,DualTSR展现出优异的感知质量和文本保真度。
原文摘要 · Abstract (English)
Scene Text Image Super-Resolution (STISR) aims to restore high-resolution details in low-resolution text images, which is crucial for both human readability and machine recognition. Existing methods, however, often depend on external Optical Character Recognition (OCR) models for textual priors or rely on complex multi-component architectures that are difficult to train and reproduce. In this paper, we introduce DualTSR, a unified end-to-end framework that addresses both issues. DualTSR employs a single multimodal transformer backbone trained with a dual diffusion objective. It simultaneously models the continuous distribution of high-resolution images via Conditional Flow Matching and the discrete distribution of textual content via discrete diffusion. This shared design enables visual and textual information to interact at every layer, allowing the model to infer text priors internally instead of relying on an external OCR module. Compared with prior multi-branch diffusion systems, DualTSR offers a simpler end-to-end formulation with fewer hand-crafted components. Experiments on synthetic Chinese benchmarks and a curated real-world evaluation protocol show that DualTSR achieves strong perceptual quality and text fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。