对比两种数据检索方式,发现结构化元数据仍对可靠数据获取至关重要。
Do Data Agents Need Semantic Metadata? A Comparative Study in Agentic Data Retrieval

- 用大模型评估两种检索方式:一个依赖网页文本,一个使用语义元数据
- 带元数据的检索精度高46.6%,错误率低20.1%的冗余页面
- 适合需要精准数据执行的自动化工作流开发者
在自主代理时代,机器可操作的数据对数据驱动流程至关重要。尽管过去十多年中,schema.org等语义元数据支撑了数据的可发现性、可访问性、互操作性和可重用性(FAIR原则),并推动了Google Dataset Search等发现工具的发展,但具备导航非结构化网络能力的大语言模型(LLMs)引发了一个根本问题:在代理式数据发现中,语义元数据是否依然必要?我们对两种环境下的代理式数据检索进行了对比分析:基线代理在数十亿开放网页中搜索,而语义代理则利用包含9000万数据集的语义元数据资源库(schema.org)。通过部署“大模型作为裁判”的评估流程,直接映射至FAIR原则,评估所获数据的语义相关性、可访问性及计算实用性。结果显示显著差异:语义代理在元数据丰富的注册表中实现44.9%更高的精度,在可机器读取下载的页面上实现46.6%更高的精度;而基线代理常出现“最后一公里”实用性失败,20.1%的结果为文字密集页,8.5%为门户首页,而非实际数据页。尽管基线代理能回答40%更多的问题,其整体精确度仍比语义代理低65.7%。结论是:虽然无结构检索适用于广泛探索任务,但结构化生态系统仍是可靠、执行导向的自主工作流不可或缺的基础。
原文摘要 · Abstract (English)
In the era of autonomous agents, machine-actionable data is critical for data-driven workflows. For more than a decade, semantic metadata like schema$.$org has anchored the FAIR principles (Findable, Accessible, Interoperable, and Reusable) for machine-actionable data and enabled discovery tools like Google Dataset Search. However, the rise of Large Language Models (LLMs) capable of navigating the unstructured web raises a fundamental question: Is semantic metadata still necessary for agentic data discovery, or can agents reliably retrieve actionable data directly from the web? We present a comparative analysis of agentic data retrieval across two distinct environments: a Baseline Agent searching billions of open-web documents, and a Semantic Agent leveraging a corpus of 90 million datasets using schema$.$org. We deploy an "LLM-as-a-judge" evaluation pipeline, mapped directly to the FAIR principles, to assess the semantic relevance, data accessibility, and computational utility of the retrieved data. Our results reveal a clear divergence. The Semantic Agent excels at retrieving actionable data, achieving a 44.9% higher precision for metadata-rich registries and a 46.6% higher precision for pages with machine-readable downloads among its returned results. Conversely, the Baseline Agent frequently suffers "Last-Mile Utility" failures, retrieving prose-heavy pages (20.1% of results) and portal landing pages (8.5%) rather than actual data pages. While the Baseline Agent achieves higher coverage by answering 40% more questions, the Semantic Agent delivers greater accuracy, achieving 65.7% higher overall precision in retrieving FAIR-compliant datasets. We conclude that while unstructured retrieval supports broad exploratory tasks, structured ecosystems remain the indispensable foundation for reliable, execution-oriented autonomous workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。