arXiv:2412.03814cs.CV2024-12被引 3

构建真实与合成图像混合数据集,提升图像修复模型训练统一性与性能。

Exploring Real&Synthetic Dataset and Linear Attention in Image Restoration

  • 提出ReSyn数据集,融合真实与AIGC图像,按复杂度均衡分布
  • 设计统一训练标准,明确迭代次数与配置,提升模型收敛可比性
  • 引入线性注意力结构RWKV-IR,支持全局与局部感知,适合大图修复

图像修复旨在从退化输入中恢复高质量图像,近年深度学习显著提升了性能。然而现有方法缺乏统一的训练基准,且常用训练与测试数据集在图像复杂度分布上存在偏差,导致修复效果不佳。为此,我们提出大规模图像修复数据集ReSyn,采用基于图像复杂度的新过滤方法,确保数据分布均衡,并包含真实图像与AIGC合成图像。同时建立统一训练标准,明确模型迭代次数与配置,聚焦于模型收敛性与修复能力评估。此外,通过将线性复杂度的RWKV集成到Transformer结构中,提出RWKV-IR模型,用深度卷积移位替代原Q-Shift以更好建模局部依赖,结合双向注意力实现全面线性注意力。还引入跨向双向RWKV模块,融合不同扫描顺序的双双向模块,实现水平与垂直注意力平衡。大量实验验证了RWKV-IR的有效性。

原文摘要 · Abstract (English)

Image restoration (IR) aims to recover high-quality images from degraded inputs, with recent deep learning advancements significantly enhancing performance. However, existing methods lack a unified training benchmark for iterations and configurations. We also identify a bias in image complexity distributions between commonly used IR training and testing datasets, resulting in suboptimal restoration outcomes. To address this, we introduce a large-scale IR dataset called ReSyn, which employs a novel image filtering method based on image complexity to ensure a balanced distribution and includes both real and AIGC synthetic images. We establish a unified training standard that specifies iterations and configurations for image restoration models, focusing on measuring model convergence and restoration capability. Additionally, we enhance transformer-based image restoration models using linear attention mechanisms by proposing RWKV-IR, which integrates linear complexity RWKV into the transformer structure, allowing for both global and local receptive fields. Instead of directly using Vision-RWKV, we replace the original Q-Shift in RWKV with a Depth-wise Convolution shift to better model local dependencies, combined with Bi-directional attention for comprehensive linear attention. We also introduce a Cross-Bi-WKV module that merges two Bi-WKV modules with different scanning orders for balanced horizontal and vertical attention. Extensive experiments validate the effectiveness of our RWKV-IR model.

图像修复数据集线性注意力AIGC

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。