arXiv:2606.18101cs.AI2026-06

通过质量感知机制提升GUI定位中的教师信号可靠性

Trust the Right Teacher: Quality-Aware Self-Distillation for GUI Grounding

论文配图:Trust the Right Teacher: Quality-Aware Self-Distillation for GUI Grounding
图 1 · 摘自论文原文
  • 引入软正确性门控与教师置信度校准,动态过滤低质量教师信号
  • 在6个基准上均超越基线模型,显著提升坐标定位精度
  • 适合需要高精度视觉语言定位的UI理解任务

图形用户界面(GUI)定位要求视觉-语言模型(VLMs)在高分辨率截图中识别微小目标元素并预测精确屏幕坐标。基于策略的自蒸馏(OPSD)是一种有前景的后训练方法,因其能提供超越硬坐标标签的密集标记级教师信号。然而,朴素的OPSD不适用于GUI定位:当学生生成的前缀已偏离目标坐标时,教师的坐标标记信号质量会下降,导致不可靠的教师信号。为此,我们提出质量感知自蒸馏方法,通过软正确性门控和教师概率缩放提升坐标标记教师信号质量。软正确性门控判断教师当前坐标标记是否仍可在学生生成前缀下完成至真实框;若不能,则降低对应信号权重。教师概率缩放则利用教师置信度作为轻量因子,进一步校准经门控后的监督强度。关键发现是:单独使用任一组件均无法提升整体性能,而两者结合始终有效。这表明二者互补:正确性门控抑制不可靠监督,教师概率缩放校准剩余信号强度。六项基准实验表明,该方法一致提升基础模型性能,并优于强基线。

原文摘要 · Abstract (English)

Graphical user interface (GUI) grounding requires vision-language models (VLMs) to identify small target elements in high-resolution screenshots and predict precise screen coordinates. On-policy self-distillation (OPSD) is a promising post-training approach for this coordinate-sensitive task, since it provides dense token-level teacher signals beyond hard coordinate labels. However, naive OPSD is not well suited to GUI grounding: OPSD evaluates the teacher on student-generated prefixes, the quality of coordinate-token teacher signals can degrade when the prefix has already deviated from the target coordinate, leading to unreliable teacher signal. To mitigate this, We propose quality-aware self-distillation for VLM-based GUI grounding, which improves coordinate-token teacher-signal quality through soft correctness-aware gating and teacher-probability scaling. The soft correctness-aware gate checks whether the teacher's current coordinate-token prediction can still be completed into the ground-truth box under the student-generated prefix. If not, the corresponding teacher signal is down-weighted. Teacher-probability scaling then uses the teacher's confidence as a lightweight factor to further calibrate the strength of the gated supervision. A key empirical finding is that neither component alone improves overall performance, whereas combining them consistently improves performance. This suggests that the two mechanisms play complementary roles: correctness-aware gating suppresses unreliable coordinate-token supervision, while teacher-probability scaling calibrates the strength of the remaining signals. Experiments across six GUI grounding benchmarks show that our method consistently improves the base model and outperforms strong baselines.

GUI定位自蒸馏视觉语言

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。