arXiv:2604.25408cs.CV2026-04

提出新评估方法,检测图像处理后语义是否改变

Beyond Fidelity: Semantic Similarity Assessment in Low-Level Image Processing

论文配图:Beyond Fidelity: Semantic Similarity Assessment in Low-Level Image Processing
图 1 · 摘自论文原文
  • 用实体与关系建模图像语义,区分前景背景
  • 在COCO和SPA-Data上优于现有指标,能捕捉语义渐变
  • 适合评估生成模型、去噪等可能改写内容的任务

低层图像处理长期依赖视觉保真度评估,但深度学习与生成模型导致图像保真度高却语义失真,传统图像质量评估(IQA)已不适用。本文将语义相似性正式定义为低层图像处理的新评估任务,旨在衡量处理前后语义内容的保持程度。基于语义实体及其关系的结构化建模,提出三元组语义相似性评分(T3S),通过提取语义实体、分离前景与背景、建模开放世界类别与关系来实现。在COCO和SPA-Data数据集上的实验表明,T3S在多种退化条件下持续优于现有保真度指标和代表性语义基准,更准确反映语义渐进变化。结果凸显了现代低层视觉中语义评估的重要性。

原文摘要 · Abstract (English)

Low-level image processing has long been evaluated mainly from the perspective of visual fidelity. However, with the rise of deep learning and generative models, processed images may preserve perceptual quality while altering semantic content, making conventional Image Quality Assessment (IQA) insufficient for semantic-level assessment. In this paper, we formalize \textit{Semantic Similarity} as a new evaluation task for low-level image processing, aimed at measuring whether semantic content is preserved after processing. We further present a structured formulation of image semantics based on semantic entities and their relations, and discuss the desired properties and constraints of a valid semantic similarity index. Based on this formulation, we propose Triplet-based Semantic Similarity Score (T3S), which models image semantics through foreground entities, background entities, and relations. T3S combines semantic entity extraction, foreground-background disentanglement, and open-world class/relation modeling. Experiments on COCO and SPA-Data show that T3S consistently outperforms existing fidelity-oriented metrics and representative semantic-level baselines, while better reflecting progressive semantic changes under diverse degradations. These results highlight the importance of semantic assessment in modern low-level vision.

语义评估图像处理生成模型COCO

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。