用颜色预训练+残差上采样提升图像超分辨率性能
Two-Stage Vision Transformer for Image Restoration: Colorization Pretraining + Residual Upsampling
- 先自监督颜色预训练，再微调做4倍超分辨率
- 在DIV2K上达PSNR 22.90 dB、SSIM 0.712
- 适合追求高质量图像重建的研究与应用
在计算机视觉中，单图超分辨率（SISR）仍是难题。本文提出ViT-SR，一种基于两阶段训练策略的Vision Transformer新方法。模型通过在色彩化任务上的自监督预训练，从数据中学习丰富且可迁移的视觉表征。随后在4倍超分辨率任务上微调，通过预测高频残差图叠加于双三次插值初始图，简化了残差学习。该方法在DIV2K基准数据集上实现PSNR 22.90 dB和SSIM 0.712的优异性能，验证了两阶段策略的有效性，展示了自监督预训练在复杂图像恢复任务中的潜力。未来可通过更大规模ViT或不同预训练任务进一步优化。
原文摘要 · Abstract (English)
In computer vision, Single Image Super-Resolution (SISR) is still a difficult problem. We present ViT-SR, a new technique to improve the performance of a Vision Transformer (ViT) employing a two-stage training strategy. In our method, the model learns rich, generalizable visual representations from the data itself through a self-supervised pretraining phase on a colourization task. The pre-trained model is then adjusted for 4x super-resolution. By predicting the addition of a high-frequency residual image to an initial bicubic interpolation, this design simplifies residual learning. ViT-SR, trained and evaluated on the DIV2K benchmark dataset, achieves an impressive SSIM of 0.712 and PSNR of 22.90 dB. These results demonstrate the efficacy of our two-stage approach and highlight the potential of self-supervised pre-training for complex image restoration tasks. Further improvements may be possible with larger ViT architectures or alternative pretext tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。