用大模型实现自然语言分析海量非结构化文档，准确率超RAG。
The Design of an LLM-powered Unstructured Analytics System
- 通过自然语言提问，系统自动规划并执行语义分析流程。
- 在NTSB真实报告上，分析准确率高于传统RAG方法。
- 提供可解释界面，帮助用户理解推理过程，增强可信度。
大语言模型具备处理非结构化数据的惊人能力，有望超越搜索，实现大规模语义分析。本文介绍名为Aryn的非结构化数据分析系统的设计理念与使用场景。用户以自然语言提出查询，系统自动生成语义计划并从大量非结构化文档中计算答案。Aryn核心是Sycamore——一个声明式文档处理引擎，提供可靠的分布式抽象DocSets，支持大规模复杂文档的分析、增强与转换。系统包含Luna查询规划器，将自然语言转化为Sycamore脚本；以及DocParse，用于解析原始PDF和文档图像，生成可处理的DocSets。实验表明，在来自美国国家运输安全委员会（NTSB）的真实报告上，Aryn的分析准确率优于RAG。鉴于当前LLM的局限性，作者强调分析系统必须具备可解释性才具实用性，并展示Aryn如何通过用户界面实现透明推理，以建立信任。
原文摘要 · Abstract (English)
LLMs demonstrate an uncanny ability to process unstructured data, and as such, have the potential to go beyond search and run complex, semantic analyses at scale. We describe the design of an unstructured analytics system, Aryn, and the tenets and use cases that motivate its design. With Aryn, users specify queries in natural language and the system automatically determines a semantic plan and executes it to compute an answer from a large collection of unstructured documents. At the core of Aryn is Sycamore, a declarative document processing engine, that provides a reliable distributed abstraction called DocSets. Sycamore allows users to analyze, enrich, and transform complex documents at scale. Aryn includes Luna, a query planner that translates natural language queries to Sycamore scripts, and DocParse, which takes raw PDFs and document images, and converts them to DocSets for downstream processing. We show how these pieces come together to achieve better accuracy than RAG on analytics queries over real world reports from the National Transportation Safety Board (NTSB). Also, given current limitations of LLMs, we argue that an analytics system must provide explainability to be practical, and show how Aryn's user interface does this to help build trust.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。