轻量级超分辨率模型,手机端秒级处理4K图像
PocketSR: The Super-Resolution Expert in Your Pocket Mobiles
- 用极简结构替代原复杂VAE,参数减少97.5%
- 0.8秒处理4K图像,速度远超同类模型
- 适合移动端部署,性能媲美顶尖多步模型
真实世界图像超分辨率(RealSR)旨在提升手机等设备拍摄的自然图像质量。现有基于大型生成模型的方法虽效果出色,但计算开销大、延迟高,难以在边缘设备部署。本文提出PocketSR,一种超轻量级单步模型,将生成建模能力引入RealSR同时保持高保真度。设计了LiteED,作为SD中耗时的VAE的高效替代,参数量减少97.5%,仍保留高质量编码解码能力。提出在线退火剪枝策略优化U-Net,逐步将生成先验从重型模块迁移至轻量模块,实现有效知识传递并进一步提升效率。为缓解剪枝导致的先验信息丢失,引入多层特征蒸馏损失。深入分析各组件设计,为未来研究提供洞见。PocketSR模型仅146M参数,处理4K图像仅需0.8秒,相比以往方法显著提速,性能与当前最优单步及多步模型相当,是边缘设备应用的理想方案。
原文摘要 · Abstract (English)
Real-world image super-resolution (RealSR) aims to enhance the visual quality of in-the-wild images, such as those captured by mobile phones. While existing methods leveraging large generative models demonstrate impressive results, the high computational cost and latency make them impractical for edge deployment. In this paper, we introduce PocketSR, an ultra-lightweight, single-step model that brings generative modeling capabilities to RealSR while maintaining high fidelity. To achieve this, we design LiteED, a highly efficient alternative to the original computationally intensive VAE in SD, reducing parameters by 97.5% while preserving high-quality encoding and decoding. Additionally, we propose online annealing pruning for the U-Net, which progressively shifts generative priors from heavy modules to lightweight counterparts, ensuring effective knowledge transfer and further optimizing efficiency. To mitigate the loss of prior knowledge during pruning, we incorporate a multi-layer feature distillation loss. Through an in-depth analysis of each design component, we provide valuable insights for future research. PocketSR, with a model size of 146M parameters, processes 4K images in just 0.8 seconds, achieving a remarkable speedup over previous methods. Notably, it delivers performance on par with state-of-the-art single-step and even multi-step RealSR models, making it a highly practical solution for edge-device applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。