arXiv:2410.23577eess.IVcs.AI2024-10中稿 · WACV 2025

用生物视觉机制设计图像重建监督信号,提升真实感与细节。

MS-Glance: Bio-Insipred Non-semantic Context Vectors and their Applications in Supervising Image Reconstruction

  • 基于人类视觉快速感知原理,构造全局与局部非语义上下文向量。
  • 在隐式神经表示与欠采样MRI重建中,均优于现有损失函数。
  • 适合需要高质量重建的医学影像与自然图像任务。

非语义上下文信息对视觉识别至关重要,因为人类视觉系统会先通过全局统计特征快速处理场景,再识别具体物体。然而,在图像重建等计算机视觉任务中,尽管语义信息被越来越多地利用,全局空间结构等非语义信息常被忽略。为此,我们提出一种受生物学启发的非语义上下文描述符——MS-Glance,以及用于比较两幅图像的注视指数(Glance Index)。全局注视向量通过感知驱动规则随机抽取像素生成,表征非语义全局上下文;局部注视向量为局部图像窗口展开,模拟聚焦观察。注视指数定义为两组标准化注视向量的内积。我们在两种重建任务中评估了引入注视监督的效果:基于隐式神经表示(INR)的图像拟合与欠采样MRI重建。大量实验表明,MS-Glance在自然图像与医学图像上均显著优于现有图像恢复损失函数。代码已公开于https://github.com/Z7Gao/MSGlance。

原文摘要 · Abstract (English)

Non-semantic context information is crucial for visual recognition, as the human visual perception system first uses global statistics to process scenes rapidly before identifying specific objects. However, while semantic information is increasingly incorporated into computer vision tasks such as image reconstruction, non-semantic information, such as global spatial structures, is often overlooked. To bridge the gap, we propose a biologically informed non-semantic context descriptor, \textbf{MS-Glance}, along with the Glance Index Measure for comparing two images. A Global Glance vector is formulated by randomly retrieving pixels based on a perception-driven rule from an image to form a vector representing non-semantic global context, while a local Glance vector is a flattened local image window, mimicking a zoom-in observation. The Glance Index is defined as the inner product of two standardized sets of Glance vectors. We evaluate the effectiveness of incorporating Glance supervision in two reconstruction tasks: image fitting with implicit neural representation (INR) and undersampled MRI reconstruction. Extensive experimental results show that MS-Glance outperforms existing image restoration losses across both natural and medical images. The code is available at \url{https://github.com/Z7Gao/MSGlance}.

图像重建生物启发非语义信息医学影像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。