通过像素分类提升屏幕内容超分辨率,速度更快、效果更好。
PixelSR: Efficient Screen Content Super-Resolution via Pixel Classification

- 训练时按像素分组计算内容注意力,增强非局部感知能力。
- 测试时利用重复性,缓存唯一像素预测结果,提速3倍以上。
- 适合需要快速处理屏幕文本和图形的实时应用。
屏幕内容图像通常由文字和图形构成,与自然图像相比,这类人为生成的图像包含大量锐利且重复的结构。然而,现有的屏幕内容超分辨率方法未能充分利用其特殊特征,模型性能和推理速度仍有较大提升空间。本文提出PixelSR,一种简单高效的方法,在保持高性能的同时显著提升推理速度。训练阶段,通过像素分箱对像素进行分类,并在每组内聚合像素特征以计算内容注意力,为每个像素引入依赖内容的非局部感受野。测试阶段,利用屏幕内容的自重复性和冗余性加速推理:将目标高分辨率像素分为三类——唯一像素、重复像素和背景像素。对唯一像素进行常规网络处理并缓存在实时查找表中;重复像素直接从表中检索结果,无需重新计算;背景像素则采用最近邻算法生成。查找表在处理完每张图像后清空并重用。实验表明,PixelSR在屏幕内容超分辨率任务中达到当前最优性能,且推理时间更短。
原文摘要 · Abstract (English)
Screen content images are generally composed of texts and graphics. Compared to natural images, these man-made images contain a large quantity of sharp but repetitive structures. However, existing works in screen content super-resolution underutilize the special characteristics of screen content, leaving a large room to improve model performance and speed up. In this paper, we propose PixelSR, a simple yet effective method to improve super-resolution performance but with faster inference speed. To improve model performance, we classify pixels via pixel binning to compute content attention in the training phase. Specifically, after binning pixels into content-dependent groups, content attention is aggregated from pixel features within each group to introduce a content-dependent and non-local receptive field for every pixel. In the testing phase, we utilize the properties of self-repetitiveness and redundancy in screen content to speed up inference without the loss of model performance. We divide targeted high-resolution pixels into three types, which are unique pixels, repeated pixels, and background pixels for each test image. We conduct conventional network processing on unique pixels and cache their predictions in the on-the-fly lookup table. For repeated pixels which have appeared in unique pixels, we directly retrieve prediction results from the lookup table without network processing. For background pixels, we use the nearest neighbor algorithm to generate high-resolution pixels. The on-the-fly lookup table is cleaned and repeats the procedure above for the next test image. Experiments show our PixelSR achieves state-of-the-art performance with shorter inference time in screen content super-resolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。