用热力图揭示文本指令如何影响图像编辑模型，提升可解释性。
Mapping the Mind of an Instruction-based Image Editing using SMILE
- 提出无需依赖模型结构的局部可解释方法，生成文本影响热力图。
- 在多个图像编辑模型上验证，显著提升透明度与可靠性。
- 适合关注AI信任、安全应用的开发者与研究者。
尽管基于指令的图像编辑模型在生成高质量图像方面取得进展，但其常被视为黑箱，阻碍了透明度与用户信任。为此，我们提出SMILE（Statistical Model-agnostic Interpretability with Local Explanations），一种新型的模型无关局部可解释性方法，通过可视化热力图揭示文本元素对图像生成模型的影响。我们将该方法应用于Pix2Pix、Image2Image-turbo和Diffusers-Inpaint等多类指令式图像编辑模型，证明其可有效提升可解释性与可靠性。同时，采用稳定性、准确性、保真度与一致性指标进行评估。结果表明，模型无关可解释性在医疗与自动驾驶等关键场景中具有巨大潜力，推动可信赖图像编辑模型的发展。
原文摘要 · Abstract (English)
Despite recent advancements in Instruct-based Image Editing models for generating high-quality images, they are known as black boxes and a significant barrier to transparency and user trust. To solve this issue, we introduce SMILE (Statistical Model-agnostic Interpretability with Local Explanations), a novel model-agnostic for localized interpretability that provides a visual heatmap to clarify the textual elements' influence on image-generating models. We applied our method to various Instruction-based Image Editing models like Pix2Pix, Image2Image-turbo and Diffusers-Inpaint and showed how our model can improve interpretability and reliability. Also, we use stability, accuracy, fidelity, and consistency metrics to evaluate our method. These findings indicate the exciting potential of model-agnostic interpretability for reliability and trustworthiness in critical applications such as healthcare and autonomous driving while encouraging additional investigation into the significance of interpretability in enhancing dependable image editing models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。