提出无需训练的高效界面导航方法,解决高分辨率截图冗余问题。
Spatio-Temporal Token Pruning for Efficient High-Resolution GUI Agents
- 通过动态降采样和分层结构感知剪枝,减少历史与空间冗余。
- 在Qwen2-VL-2B上实现3.4倍算力降低,速度提升3.3倍,性能保留超94%。
- 适合资源受限环境下的实时高精度图形界面交互任务。
纯视觉图形用户界面(GUI)代理虽具通用交互能力,但因高分辨率屏幕截图与历史轨迹中存在大量时空冗余,导致严重效率瓶颈。现有压缩范式存在两大关键错位:时间上的不匹配——统一的历史编码违背了代理的“记忆衰退”注意力模式;空间拓扑冲突——无结构剪枝破坏网格完整性,引发空间幻觉。为此,我们提出GUIPruner,一种无需训练的高分辨率GUI导航框架。其融合时序自适应分辨率(TAR),通过衰减式重采样消除历史冗余;以及分层结构感知剪枝(SSP),优先保留交互前景与语义锚点,同时保护全局布局。跨多种基准的广泛评估表明,GUIPruner持续达成当前最优性能,有效防止大规模模型在高压缩下的崩溃。特别地,在Qwen2-VL-2B上,该方法实现3.4倍FLOPs降低与3.3倍视觉编码延迟加速,同时保持超过94%的原始性能,实现低资源消耗下的实时、高精度导航。
原文摘要 · Abstract (English)
Pure-vision GUI agents provide universal interaction capabilities but suffer from severe efficiency bottlenecks due to the massive spatiotemporal redundancy inherent in high-resolution screenshots and historical trajectories. We identify two critical misalignments in existing compression paradigms: the temporal mismatch, where uniform history encoding diverges from the agent's "fading memory" attention pattern, and the spatial topology conflict, where unstructured pruning compromises the grid integrity required for precise coordinate grounding, inducing spatial hallucinations. To address these challenges, we introduce GUIPruner, a training-free framework tailored for high-resolution GUI navigation. It synergizes Temporal-Adaptive Resolution (TAR), which eliminates historical redundancy via decay-based resizing, and Stratified Structure-aware Pruning (SSP), which prioritizes interactive foregrounds and semantic anchors while safeguarding global layout. Extensive evaluations across diverse benchmarks demonstrate that GUIPruner consistently achieves state-of-the-art performance, effectively preventing the collapse observed in large-scale models under high compression. Notably, on Qwen2-VL-2B, our method delivers a 3.4x reduction in FLOPs and a 3.3x speedup in vision encoding latency while retaining over 94% of the original performance, enabling real-time, high-precision navigation with minimal resource consumption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。