为智能体设计专用网页组件接口，提升自动化任务成功率
CI4A: Semantic Component Interfaces for Agents Empowering Web Automation
- 构建语义化组件接口，将复杂网页操作抽象为统一工具
- 在WebArena基准上达成86.3%任务成功率，显著优于现有方法
- 适合研究智能体交互、网页自动化与人机界面优化的开发者
尽管大语言模型在高层语义规划方面表现优异，但在细粒度网页组件操作上仍显不足。现有研究多通过强化学习增强模型的上下文感知能力，但本工作提出反向思路：不强迫智能体适应人类界面，而是为智能体构建专用交互接口。本文提出组件接口（CI4A），将网页组件的复杂交互逻辑封装为统一的工具原语，供智能体调用。我们在工业级前端框架Ant Design中实现了该机制，覆盖23类常用UI组件。同时开发了动态动作空间的混合智能体，能根据页面状态灵活调用可用的CI4A工具。基于集成CI4A的Ant Design，我们重构并升级了WebArena基准以评估当前最先进方法。实验表明，基于CI4A的智能体显著超越现有方案，任务成功率达到86.3%，执行效率也大幅提升。
原文摘要 · Abstract (English)
While Large Language Models demonstrate remarkable proficiency in high-level semantic planning, they remain limited in handling fine-grained, low-level web component manipulations. To address this limitation, extensive research has focused on enhancing model grounding capabilities through techniques such as Reinforcement Learning. However, rather than compelling agents to adapt to human-centric interfaces, we propose constructing interaction interfaces specifically optimized for agents. This paper introduces Component Interface for Agent (CI4A), a semantic encapsulation mechanism that abstracts the complex interaction logic of UI components into a set of unified tool primitives accessible to agents. We implemented CI4A within Ant Design, an industrial-grade front-end framework, covering 23 categories of commonly used UI components. Furthermore, we developed a hybrid agent featuring an action space that dynamically updates according to the page state, enabling flexible invocation of available CI4A tools. Leveraging the CI4A-integrated Ant Design, we refactored and upgraded the WebArena benchmark to evaluate existing SoTA methods. Experimental results demonstrate that the CI4A-based agent significantly outperforms existing approaches, achieving a new SoTA task success rate of 86.3%, alongside substantial improvements in execution efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。