测试视觉语言模型在网页界面中发现细微差异的能力，发现当前模型表现仍很有限。
DiffSpot: Can VLMs Spot Fine-Grained Visual Differences in Web Interfaces?
- 构建代码驱动的网页差异检测基准，精准控制变化区域和程度。
- 13个前沿模型零样本测试，最高识别率仅40.7%，难题难度下不足23%。
- 差异难易程度与像素变化量无关，揭示模型感知机制缺陷，适合界面智能研究者。
视觉语言模型（VLMs）在高层图像-文本对齐上取得显著进展，但在感知细微视觉差异方面能力仍受限。本文研究渲染后的网页界面中的局部视觉变化，这类变化既是细粒度感知的诊断测试，也是GUI代理与设计工具的实际需求。我们提出DiffSpot，一个面向开放性找不同任务的代码驱动基准。DiffSpot通过修改单个目标元素的CSS属性，自包含HTML重渲染页面，并记录变化属性、元素及变化幅度，通过定位门筛选仅目标元素有像素差异的图像对。基准包含4,400对样本，其中3,900对为含差异对，覆盖13种CSS属性操作符与三个难度层级，500对无差异对用于控制幻觉。零样本评估13个前沿VLM，最佳模型仅能识别40.7%的真实变化，所有模型在难题层级召回率均低于23%。结果还显示，难度高度依赖具体属性，像素变化量与CLIP距离均无法有效预测召回率。
原文摘要 · Abstract (English)
Vision-language models (VLMs) have made strong progress on high-level image-text alignment, yet their ability to perceive subtle visual differences remains limited. We study this problem in rendered web interfaces, where localized visual changes are both a diagnostic test of fine-grained perception and a practical requirement for GUI agents and design tools. We introduce \textbf{DiffSpot}, a code-driven benchmark for open-ended spot-the-difference on web interfaces. DiffSpot constructs controlled image pairs by mutating a single CSS property of a target element in self-contained HTML, re-rendering the page, and recording the changed property, element, and mutation magnitude. A grounding gate retains only pairs whose rendered pixel difference is confined to the target element. The benchmark contains 4{,}400 pairs, including 3{,}900 has-diff pairs balanced across 13 CSS-property operators and three difficulty tiers, plus 500 no-diff pairs for hallucination control. Evaluating 13 frontier VLMs zero-shot, we find that even the best model identifies only $40.7\%$ of true changes, with Hard-tier Recall below $23\%$ for every model. DiffSpot further shows that difficulty is strongly property-dependent: across CSS operators, neither pixel magnitude nor CLIP distance reliably predicts Recall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。