arXiv:2603.18447cs.DBcs.AI2026-03

让AI自动从网页挖数据并建可查数据库,准确率超90%。

SODIUM: From Open Web Data to Queryable Databases

  • 多智能体协作探索网页,用新算法系统提取信息
  • 在105个任务上达91.1%准确率,比最强基线高近2倍
  • 适合需要跨源数据整合的研究者或自动化分析场景

研究人员常需整合多个网络来源的数据来回答分析问题,但手动搜索、提取和整理耗时巨大。本文将此过程形式化为SODIUM任务,即把开放网络视为潜在数据库,需系统化构建以支持查询。解决SODIUM需三步:深入专业地探索开放网络;利用结构关联实现系统性信息抽取;将数据整合为连贯可查询的数据库实例。为此,我们构建了涵盖6个领域的105个任务的SODIUM-Bench基准,评估发现现有6个先进AI代理在该任务上最高仅达46.5%准确率。为此,我们提出SODIUM-Agent,由网页探索者与缓存管理器组成的多智能体系统,采用ATP-BFS算法,并通过优化缓存源与导航路径进行训练,实现深度全面的网络探索与结构一致的信息抽取。SODIUM-Agent在SODIUM-Bench上达到91.1%准确率,较最强基线提升约2倍,最弱基线提升高达73倍。

原文摘要 · Abstract (English)

During research, domain experts often ask analytical questions whose answers require integrating data from a wide range of web sources. Thus, they must spend substantial effort searching, extracting, and organizing raw data before analysis can begin. We formalize this process as the SODIUM task, where we conceptualize open domains such as the web as latent databases that must be systematically instantiated to support downstream querying. Solving SODIUM requires (1) conducting in-depth and specialized exploration of the open web, which is further strengthened by (2) exploiting structural correlations for systematic information extraction and (3) integrating collected information into coherent, queryable database instances. To quantify the challenges in automating SODIUM, we construct SODIUM-Bench, a benchmark of 105 tasks derived from published academic papers across 6 domains, where systems are tasked with exploring the open web to collect and aggregate data from diverse sources into structured tables. Existing systems struggle with SODIUM tasks: we evaluate 6 advanced AI agents on SODIUM-Bench, with the strongest baseline achieving only 46.5% accuracy. To bridge this gap, we develop SODIUM-Agent, a multi-agent system composed of a web explorer and a cache manager. Powered by our proposed ATP-BFS algorithm and optimized through principled management of cached sources and navigation paths, SODIUM-Agent conducts deep and comprehensive web exploration and performs structurally coherent information extraction. SODIUM-Agent achieves 91.1% accuracy on SODIUM-Bench, outperforming the strongest baseline by approximately 2 times and the weakest by up to 73 times.

数据挖掘多智能体自动化探索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。