arXiv:2511.06087cs.CVcs.AI2025-11被引 1

融合CNN与视觉Transformer,提升模糊文字图像恢复效果。

Hybrid CNN-ViT Framework for Motion-Blurred Scene Text Restoration

  • 用CNN提取局部特征,用ViT建模全局上下文关系。
  • 在合成数据上训练,达到32.20 dB PSNR和0.934 SSIM。
  • 模型轻量高效,适合实时应用,如自动驾驶场景。

场景文本图像中的运动模糊严重影响可读性,制约自动驾驶、文档数字化与视觉信息检索等任务的可靠性。传统去模糊方法难以处理空间变化的模糊,且缺乏对长距离依赖的建模能力。为此,我们提出一种融合卷积神经网络(CNN)与视觉变压器(ViT)的混合深度学习框架,兼顾局部特征提取与全局上下文推理。采用基于CNN的编码器-解码器结构以保留结构细节,同时引入变换器模块通过自注意力机制增强全局感知。训练使用从TextOCR构建的配对数据集,其中清晰文本样本与多尺寸、多方向的真实运动模糊核生成的模糊版本配对。优化采用包含平均绝对误差(MAE)、均方误差(MSE)、感知相似性和结构相似性(SSIM)的复合损失函数。定量评估显示,该方法在PSNR上达32.20 dB,SSIM为0.934,参数量仅283万,平均推理时间61毫秒。结果验证了该混合设计的有效性与计算效率,具备实际应用潜力。

原文摘要 · Abstract (English)

Motion blur in scene text images severely impairs readability and hinders the reliability of computer vision tasks, including autonomous driving, document digitization, and visual information retrieval. Conventional deblurring approaches are often inadequate in handling spatially varying blur and typically fall short in modeling the long-range dependencies necessary for restoring textual clarity. To overcome these limitations, we introduce a hybrid deep learning framework that combines convolutional neural networks (CNNs) with vision transformers (ViTs), thereby leveraging both local feature extraction and global contextual reasoning. The architecture employs a CNN-based encoder-decoder to preserve structural details, while a transformer module enhances global awareness through self-attention. Training is conducted on a curated dataset derived from TextOCR, where sharp scene-text samples are paired with synthetically blurred versions generated using realistic motion-blur kernels of multiple sizes and orientations. Model optimization is guided by a composite loss that incorporates mean absolute error (MAE), squared error (MSE), perceptual similarity, and structural similarity (SSIM). Quantitative evaluations show that the proposed method attains 32.20 dB in PSNR and 0.934 in SSIM, while remaining lightweight with 2.83 million parameters and an average inference time of 61 ms. These results highlight the effectiveness and computational efficiency of the CNN-ViT hybrid design, establishing its practicality for real-world motion-blurred scene-text restoration.

图像去模糊文本恢复CNN-ViT轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。