用可逆网络提升图像超分的细节感知,兼顾纹理与内容一致性。
MGHF: Multi-Granular High-Frequency Perceptual Loss for Image Super-Resolution
- 基于可逆神经网络设计多粒度高频感知损失,减少信息丢失。
- 在多种超分模型上实现显著性能提升,尤其增强局部细节表现。
- 适合关注真实感生成与细节保留的研究者和开发者。
尽管感知损失被广泛用于生成更真实、美观且细节丰富的高分辨率图像,但多数方法基于卷积神经网络,导致引导过程中的信息损失,并依赖复杂架构与训练流程。本文提出一种基于可逆神经网络(INN)的朴素多粒度高频感知损失(MGHF-n),在ImageNet上训练以克服上述问题。进一步构建综合框架(MGHF-c),通过多种约束实现跨视角的信息保留、优先级划分与正则化:纹理与风格保持、内容保持、区域细节保持及联合内容-风格正则化。通过自适应熵剪枝与特征重加权实现信息优先处理。采用格拉姆矩阵损失保持风格,均方误差损失保持内容。此外,提出相关性损失以实现内容-风格一致性,抑制不必要的纹理生成同时保留内容信息。由于小图像区域可能包含精细细节,我们在INN特征中引入调制的PatchNCE作为局部信息保持目标。大量实验表明,该框架在多种超分算法(包括GAN与扩散模型)上均显著提升性能。代码将在评审后公开。
原文摘要 · Abstract (English)
While different variants of perceptual losses have been employed in super-resolution literature to synthesize more realistic, appealing, and detailed high-resolution images, most are convolutional neural networks-based, causing information loss during guidance and often relying on complicated architectures and training procedures. We propose an invertible neural network (INN)-based naive \textbf{M}ulti-\textbf{G}ranular \textbf{H}igh-\textbf{F}requency (MGHF-n) perceptual loss trained on ImageNet to overcome these issues. Furthermore, we develop a comprehensive framework (MGHF-c) with several constraints to preserve, prioritize, and regularize information across multiple perspectives: texture and style preservation, content preservation, regional detail preservation, and joint content-style regularization. Information is prioritized through adaptive entropy-based pruning and reweighting of INN features. We utilize Gram matrix loss for style preservation and mean-squared error loss for content preservation. Additionally, we propose content-style consistency through correlation loss to regulate unnecessary texture generation while preserving content information. Since small image regions may contain intricate details, we employ modulated PatchNCE in the INN features as a local information preservation objective. Extensive experiments on various super-resolution algorithms, including GAN- and diffusion-based methods, demonstrate that our MGHF framework significantly improves performance. After the review process, our code will be released in the public repository.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。