首个可在移动端实时完成超高清图像修复的基准方法
RETHINED: A New Benchmark and Baseline for Real-Time High-Resolution Image Inpainting On Edge Devices
- 轻量CNN+无分辨率依赖的补丁替换，兼顾结构与细节
- 30ms内完成超高清修复，比现有方法快100倍
- 适合移动端图像修复应用，尤其注重速度与精度
现有图像修复方法在低分辨率下表现优异，但在高分辨率场景下普遍失效且依赖高性能硬件，难以部署于边缘设备。为此，我们提出首个面向边缘设备的实时高分辨率图像修复基准（RETHINED），可在多种移动设备上实现超高清图像的实时修复（≤30ms）。该方法由轻量级卷积神经网络恢复结构，并结合无分辨率依赖的补丁替换机制生成细节纹理，融合了CNN的结构建模能力与基于补丁方法的高级细节表现。我们在多款移动设备上进行广泛测试，验证了其在保持一致修复质量的同时，相较现有最先进方法提速100倍。此外，我们发布了首个自由形状掩码的超高清图像修复数据集DF8K-Inpainting。
原文摘要 · Abstract (English)
Existing image inpainting methods have shown impressive completion results for low-resolution images. However, most of these algorithms fail at high resolutions and require powerful hardware, limiting their deployment on edge devices. Motivated by this, we propose the first baseline for REal-Time High-resolution image INpainting on Edge Devices (RETHINED) that is able to inpaint at ultra-high-resolution and can run in real-time ($\leq$ 30ms) in a wide variety of mobile devices. A simple, yet effective novel method formed by a lightweight Convolutional Neural Network (CNN) to recover structure, followed by a resolution-agnostic patch replacement mechanism to provide detailed texture. Specially our pipeline leverages the structural capacity of CNN and the high-level detail of patch-based methods, which is a key component for high-resolution image inpainting. To demonstrate the real application of our method, we conduct an extensive analysis on various mobile-friendly devices and demonstrate similar inpainting performance while being $\mathrm{100 \times faster}$ than existing state-of-the-art methods. Furthemore, we realease DF8K-Inpainting, the first free-form mask UHD inpainting dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。