用共享接口图谱加速网页智能体,比浏览器自动化快3.6倍
Internal APIs Are All You Need: Shadow APIs, Shared Discovery, and the Case Against Browser-First Agent Architectures
- 构建共享路由图,从真实浏览流量中学习网站内部接口
- 缓存后接口调用平均仅100毫秒,比浏览器自动化快3.6倍
- 支持自愿参与与自纠错,适合追求效率的智能体开发者
自主智能体越来越多地与网络交互,但多数网站仍为人类浏览器设计,造成根本性不匹配。现有方法需反复浏览页面、检查DOM并逆向工程可调用接口,过程缓慢且脆弱。我们发现现代网站已通过内部API(称为影子API)暴露其功能接口。提出Unbrowse系统,将浏览器路由发现转化为对这些第一方接口的集体维护索引。系统被动学习真实浏览流量中的路由,并通过直接API调用提供缓存结果。在94个域名的单机实时网页基准测试中,完全预热的缓存执行平均耗时950毫秒,而Playwright浏览器自动化为3,404毫秒(均值提速3.6倍,中位数提速5.4倍),良好缓存路由低于100毫秒。采用三路径执行模型——本地缓存、共享图或浏览器回退——确保系统自愿且自修正。通过x402协议实现三层微支付:按查询收费(层级3)、一次性安装费(层级1)、可选执行费(层级2)。所有层级均基于理性采纳必要条件:仅当总费用低于浏览器重发现预期成本时,代理才使用共享图。
原文摘要 · Abstract (English)
Autonomous agents increasingly interact with the web, yet most websites remain designed for human browsers -- a fundamental mismatch that the emerging ``Agentic Web'' must resolve. Agents must repeatedly browse pages, inspect DOMs, and reverse-engineer callable routes -- a process that is slow, brittle, and redundantly repeated across agents. We observe that every modern website already exposes internal APIs (sometimes called \emph{shadow APIs}) behind its user interface -- first-party endpoints that power the site's own functionality. We present Unbrowse, a shared route graph that transforms browser-based route discovery into a collectively maintained index of these callable first-party interfaces. The system passively learns routes from real browsing traffic and serves cached routes via direct API calls. In a single-host live-web benchmark of equivalent information-retrieval tasks across 94 domains, fully warmed cached execution averaged 950\,ms versus 3{,}404\,ms for Playwright browser automation (3.6$\times$ mean speedup, 5.4$\times$ median), with well-cached routes completing in under 100\,ms. A three-path execution model -- local cache, shared graph, or browser fallback -- ensures the system is voluntary and self-correcting. A three-tier micropayment model via the x402 protocol charges per-query search fees for graph lookups (Tier~3), a one-time install fee for discovery documentation (Tier~1), and optional per-execution fees for site owners who opt in (Tier~2). All tiers are grounded in a necessary condition for rational adoption: an agent uses the shared graph only when the total fee is lower than the expected cost of browser rediscovery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。