提出新评估指标BPM,精准衡量图像编辑中修改与保留的平衡。
Balancing Preservation and Modification: A Region and Semantic Aware Metric for Instruction-Based Image Editing
- 分离编辑相关与无关区域,分层评估修改准确性与内容保留。
- 在多个数据集上与人工评价高度一致,优于现有指标。
- 可嵌入编辑模型提升性能,适合研究者与开发者使用。
基于指令的图像编辑旨在忠实遵循指令修改图像,同时保持无关内容不变,已取得显著进展。然而,目前缺乏全面评估编辑质量的指标。现有方法或需高昂的人工评估成本,难以大规模应用;或沿用其他任务指标,忽略任务特异性,无法兼顾修改与保留的全面评估,导致评价偏差。为此,我们提出专为指令图像编辑设计的新指标BPM,通过显式分离图像中的编辑相关与无关区域进行针对性评估。首先定位编辑相关区域,再通过两阶段过程评估:区域感知判别器判断修改区域的位置与大小是否符合指令;语义感知判别器进一步评估相关区域内指令内容一致性及无关区域的内容保留性,实现全面且可解释的评估。此外,BPM中的区域定位能力可集成至编辑模型以提升性能,展现广泛适用性。我们在多个指令编辑数据集上验证了BPM的有效性,结果表明其与人工评价的一致性最高,证明其优越性。代码已开源:https://joyli-x.github.io/BPM/
原文摘要 · Abstract (English)
Instruction-based image editing, which aims to modify the image faithfully according to the instruction while preserving irrelevant content unchanged, has made significant progress. However, there still lacks a comprehensive metric for assessing the editing quality. Existing metrics either require high human evaluation costs, which hinder large-scale evaluation, or are adapted from other tasks and lose task-specific concerns, failing to comprehensively evaluate both instruction-based modification and preservation of irrelevant regions, resulting in biased evaluation. To tackle this, we introduce a new metric called Balancing Preservation and Modification (BPM), tailored for instruction-based image editing by explicitly disentangling the image into editing-relevant and irrelevant regions for specific consideration. We first identify and locate editing-relevant regions, followed by a two-tier process to assess editing quality: Region-Aware Judge evaluates whether the position and size of the edited region align with the instruction, and Semantic-Aware Judge further assesses the instruction content compliance within editing-relevant regions as well as content preservation within irrelevant regions, yielding comprehensive and interpretable quality assessment. Moreover, the editing-relevant region localization in BPM can be integrated into image editing approaches to improve editing quality, demonstrating its broad applicability. We verify the effectiveness of the BPM metric on comprehensive instruction-editing data, and the results show the highest alignment with human evaluation compared to existing metrics, indicating its efficacy. Code is available at: https://joyli-x.github.io/BPM/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。