将界面访问树压缩为更紧凑结构，提升AI操作界面的效率与准确率。
A11y-Compressor: A Framework for Enhancing the Efficiency of GUI Agent Observations through Visual Context Reconstruction and Redundancy Reduction
- 通过检测模态、去重和语义重构，压缩线性化访问树
- 输入令牌减少至原大小的22%，任务成功率提升5.1个百分点
- 适合需要高效界面交互的AI代理开发与优化
能够与图形用户界面（GUI）交互的AI代理需要有效的观察表示以实现可靠定位。无障碍树是一种常用的基于文本的格式，用于编码UI元素属性，但存在冗余且缺乏空间关系等结构信息。我们提出A11y-Compressor框架，将线性化的无障碍树转换为紧凑且结构化的表示。其实现Compressed-a11y采用轻量级、结构化的转换流程，包含模态检测、冗余消除和语义重构。在OSWorld基准上的实验表明，Compressed-a11y将输入令牌减少至原始的22%，平均任务成功率提升5.1个百分点。
原文摘要 · Abstract (English)
AI agents that interact with graphical user interfaces (GUIs) require effective observation representations for reliable grounding. The accessibility tree is a commonly used text-based format that encodes UI element attributes, but it suffers from redundancy and lacks structural information such as spatial relationships among elements. We propose A11y-Compressor, a framework that transforms linearized accessibility trees into compact and structured representations. Our implementation, Compressed-a11y, applies a lightweight and structured transformation pipeline with modal detection, redundancy reduction, and semantic structuring. Experiments on the OSWorld benchmark show that Compressed-a11y reduces input tokens to 22% of the original while improving task success rates by 5.1 percentage points on average.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。