arXiv:2607.22708cs.CV2026-07被引 1

0.9B参数模型实现手机端高精度界面理解,兼顾速度与效果。

StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design

论文配图:StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design
图 1 · 摘自论文原文
  • 通过架构-训练-部署协同设计,优化屏幕特性的视觉编码与计算效率。
  • 在ScreenQA和中文OCRBench上超越2.3B模型,性能接近1.3B通用模型。
  • 可在骁龙8 Gen5上稳定运行,推理延迟仅0.84秒,适合移动端实际部署。

在端设备上部署具备完整界面理解能力的视觉语言模型长期面临准确率与效率的矛盾:一方面需满足OCR、屏幕理解、视觉问答和元素定位的高准确率要求;另一方面受限于移动芯片严格的算力、内存和功耗预算。现有工作或牺牲一方,或仅停留在仿真阶段。本文提出StepX-Edge,一个0.9B参数的端侧界面视觉语言模型,通过架构、训练与部署的三层协同设计解决该矛盾。架构上,采用面向界面的分层视觉编码(ULVE)和渐进式维度投影(PDP)以适配极端长宽比与细粒度感知,同时保持对主流移动端NPU操作符的原生兼容性。训练方面,基于界面子任务间的互促效应,设计五阶段分步式训练框架(StepX-Curriculum),使四项能力在紧凑参数下协同提升而非相互干扰。部署阶段,采用模块级差异化两阶段量化方案(PTQ-to-QAT),将量化后精度损失控制在1%以内。StepX-Edge在≤1B模型中达到最强整体界面理解能力,在ScreenQA(88.76 F1)和Chinese OCRBench v2(57.25)上超越所有2B–2.3B基线,并在RefCOCO(92.0%)和OCRBench v1(831)上匹配1.3B–2.3B通用视觉语言模型,参数量远少。经W4A16+KV8量化后,可在骁龙8 Gen5设备上稳定运行,首字响应时间约0.84秒,解码速度达98词/秒,峰值内存1.4 GB。我们将开源训练数据、完整训练流程及量化部署方案。

原文摘要 · Abstract (English)

Deploying a vision-language model with full UI understanding on end devices has long been trapped between accuracy and efficiency: on one side is the accuracy bar for OCR, screen understanding, visual question answering, and element grounding; on the other is the strict compute, memory, and power budget of mobile chips. Existing work either trades one for the other, or stops at simulation without real-device validation. We present StepX-Edge, a 0.9B-parameter on-device UI vision-language model that resolves this tension through three-layer co-design of architecture, training, and deployment. Architecturally, UI-aware Layered Visual Encoding (ULVE) and a Progressive Dimensionality Projection (PDP) connector target the extreme aspect ratios and fine-grained perception of screens, while standard full attention throughout ensures native compatibility with mainstream mobile NPU operators. For training, the five-stage StepX-Curriculum framework is designed around our observation of mutual-promotion effects among UI subtasks, so that all four capabilities grow synergistically under a tight parameter budget rather than interfering. For deployment, a module-wise differentiated two-stage PTQ-to-QAT quantization scheme keeps the post-quantization accuracy loss within 1%. StepX-Edge achieves the strongest overall UI understanding among <=1B models, surpassing all 2B-2.3B baselines on ScreenQA (88.76 F1) and Chinese OCRBench v2 (57.25), and matching 1.3B-2.3B general VLMs on RefCOCO (92.0%) and OCRBench v1 (831) with far fewer parameters. After W4A16+KV8 quantization, the model runs stably on Snapdragon 8 Gen5 devices with ~0.84 s TTFT, 98 tok/s decode, and 1.4 GB peak memory. We will open-source the training data, the full training recipe, and the quantization deployment pipeline.

端侧模型视觉语言界面理解量化部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。