首个专注小物体编辑的评测基准，揭示指令图像模型在此任务上的短板。
DLEBench: Evaluating Small-scale Object Editing Ability for Instruction-based Image Editing Model
- 构建涵盖7类指令的1889样本测试集，目标物占图像面积1%-10%。
- 提出双模式评估框架与细化评分标准，减少主观偏差。
- 10个模型在小物体编辑上表现显著不足，凸显评测必要性。
指令式图像编辑模型（IIEMs）虽在现有基准上表现出良好的指令遵循能力和推理能力，但其对小物体的编辑能力仍缺乏系统评估，而这一能力对精准局部编辑和细节优化至关重要。本文提出DeepLookEditBench（DLEBench），首个专注于评估IIEMs小尺度物体编辑能力的基准。该基准包含1889个样本，覆盖7类指令，目标物仅占图像面积的1%-10%，涵盖部分遮挡、多对象编辑等复杂场景。为确保评估可靠性，我们设计了细化评分标准，降低指令遵循与视觉一致性两项指标的主观性和模糊性，并引入双模式评估框架（工具驱动与理想引导模式），缓解大模型作为评判者与人类判断之间的偏差。对10个IIEMs的实证结果表明，模型在小物体编辑任务上存在显著性能差距，亟需专用评测基准以推动该能力发展。
原文摘要 · Abstract (English)
Significant progress has been made in the field of Instruction-based Image Editing Models (IIEMs). However, while these models demonstrate plausible adherence to instructions and strong reasoning ability on current benchmarks, their ability to edit small objects remains underexplored, despite its importance for precise local editing and refining details in both real and generated images. In this paper, we introduce DeepLookEditBench (DLEBench), the first benchmark dedicated to assessing the abilities of IIEMs in editing small-scale objects. Specifically, we construct a challenging testbed comprising 1889 samples across seven instruction types. In these samples, target objects occupy only 1%-10% of the image area, covering complex scenarios such as partial occlusion and multi-object editing. To ensure robust evaluation on this benchmark, we propose an evaluation protocol with refined score rubrics to minimize subjectivity and ambiguity in two criteria: Instruction Following and Visual Consistency. This protocol also introduces a dual-mode evaluation framework (Tool-driven and Oracle-guided Modes) addressing the misalignment between LMM-as-a-Judge and human judgements on DLEBench. Empirical results on 10 IIEMs reveal significant performance gaps in small-scale object editing, highlighting the need for specialized benchmarks to advance this ability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。