为智能代理设计的轻量级浏览器端数据查询引擎，让文本分析更快更省资源。
A Query Engine for the Agents
- 用JS原生库直接读取对象存储中的Parquet和Iceberg数据
- 异步按需执行，关键计算仅在需要时触发，提速300倍以上
- 适合运行在浏览器或代理沙箱中，专为AI客户端应用优化
当前生产环境中增长最快的数据是无结构文本：代理轨迹、聊天记录、推理链、模型输出。人们希望分析这些数据，但传统SQL无法处理，因文本分析必须依赖模型。如今这类分析主要在新类AI应用（如Claude Code、Cursor、Claude Desktop）中进行，它们在客户端运行，同时承载用户与LLM代理。然而，现有湖仓读取路径（如Spark、Trino、托管仓库）难以在JS运行时使用。为此，我们提出Hyperparam——三个开源JS库（Hyparquet、Squirreling、Icebird），总大小不足70KB，可直接从对象存储读取Parquet和Apache Iceberg，并通过每单元异步执行实现分析操作与模型解释的融合。Squirreling在过滤限制查询上比DuckDB-WASM快300倍，排序限制查询快192倍，十项代理分析任务成本降低三分之一。我们认为，数据工程需适应现役的AI原生客户端应用及协同工作的智能代理。
原文摘要 · Abstract (English)
The fastest-growing data in production today is unstructured text: agent traces, chat logs, reasoning chains, model outputs. People want to analyze it, and the questions worth asking ("show me where the agent got confused") cannot be answered by SQL alone, since text is not queryable without a model in the query path. The natural place this analysis is happening is the new class of AI applications (Claude Code, Cursor, Claude Desktop, in-browser agents) that run client-side and host both a human user and an LLM agent in the same process. These applications increasingly want to work with data, but the lakehouse read path has been hard to use from a JS runtime: Spark, Trino, and managed warehouses do not fit there. To build this new kind of AI data application, three properties of the engine become first-order: a JS-native distribution that drops into the runtime the application already runs in, a bundle small enough to ship inside a cold tab or per-turn agent sandbox, and a way to interleave analytic operators with model-based interpretation of text. We present Hyperparam, three open-source JavaScript libraries (Hyparquet, Squirreling, Icebird) totaling under 70 KB, that read Parquet and Apache Iceberg directly from object storage and meet the third property with per-cell, async-native SQL execution, so expensive cells fire only when downstream operators demand them. Squirreling runs LLM-shaped async UDFs over 300x faster than DuckDB-WASM on filter-bounded queries (and 192x on sort-bounded queries) and completes a ten-task agent analyst suite at two-thirds lower cost. We argue that data engineering as a discipline needs to update for the AI-native client applications now in production and the agents that work alongside their users.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。