3B参数小模型在单张显卡上训练，实现跨分辨率界面定位高精度。
ZonUI-3B: A Lightweight Vision-Language Model for Cross-Resolution GUI Grounding
- 用24K多平台截图数据+两阶段微调，解决高分辨桌面场景数据少问题。
- 在ScreenSpot和ScreenSpot-v2上分别达到84.9%和86.4%准确率，超越同类小模型。
- 适合资源有限但需高精度界面理解的开发者与研究者使用。
本文提出ZonUI-3B，一个轻量级视觉语言模型，可在单张消费级显卡（RTX 4090）上完成全部训练，同时在界面定位任务上表现媲美更大模型。模型融合多项创新：(i) 整合来自移动端、桌面端和网页端的24,000个跨平台、多分辨率界面截图数据，有效缓解高分辨率桌面环境下的数据稀缺问题；(ii) 采用两阶段微调策略，先在跨平台数据上进行通用界面理解训练，再针对高分辨率数据进行专项优化，显著提升模型适应性；(iii) 通过数据清洗与冗余降低策略，证明在减少冗余的前提下随机采样更小数据集可达到与大集相当的效果，强调数据多样性优于数据量。在ScreenSpot、ScreenSpot-v2及挑战性ScreenSpot-Pro基准上的实证评估表明，ZonUI-3B在标准任务中表现出色，准确率分别达84.9%和86.4%，超越参数量低于4B的已有模型。消融实验验证了均衡采样与两阶段微调对鲁棒性的关键作用，尤其在高分辨率桌面场景中表现突出。代码已开源：https://github.com/Han1018/ZonUI-3B。
原文摘要 · Abstract (English)
In this paper, we present ZonUI-3B, a lightweight Vision-Language Model (VLM) that can be fully trained on a single consumer-grade GPU (RTX 4090) while delivering performance comparable to significantly larger models on GUI grounding tasks. The model incorporates several key innovations: (i) combine cross-platform, multi-resolution dataset of 24K examples from diverse sources including mobile, desktop, and web GUI screenshots to effectively address data scarcity in high-resolution desktop environments; (ii) a two-stage fine-tuning strategy, where initial cross-platform training establishes robust GUI understanding, followed by specialized fine-tuning on high-resolution data to significantly enhance model adaptability; and (iii) data curation and redundancy reduction strategies, demonstrating that randomly sampling a smaller subset with reduced redundancy achieves performance comparable to larger datasets, emphasizing data diversity over sheer volume. Empirical evaluation on standard GUI grounding benchmarks, including ScreenSpot, ScreenSpot-v2, and the challenging ScreenSpot-Pro, highlights ZonUI-3B's exceptional accuracy, achieving 84.9% on ScreenSpot and 86.4% on ScreenSpot-v2, surpassing prior models under 4B parameters. Ablation studies validate the critical role of balanced sampling and two-stage fine-tuning in enhancing robustness, particularly in high-resolution desktop scenarios. The ZonUI-3B is available at: https://github.com/Han1018/ZonUI-3B
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。