用LLM生成执行图,让企业数据查询快又准。
BatchDAG: LLM-Planned Execution Graphs for Scalable Ad-Hoc Analysis Over Enterprise Data

- LLM规划操作图,支持并行执行与数据流管理
- 实体分组批量处理,减少47倍LLM调用
- 适合需要快速分析海量会议数据的团队
大语言模型(LLMs)在单文档分析上表现优异,但在企业级数据集上的跨实体分析任务中因上下文溢出、实体归因丢失和串行工具调用导致延迟过高而失效。本文提出BatchDAG系统,由LLM生成包含SQL查询、语义搜索、内存转换、并行分支和单次分析等操作的类型化有向无环图(DAG),由确定性引擎通过拓扑波并行机制与结构化JSON数据流进行评估。关键优化‘实体感知批处理’在分支前按逻辑实体分组,使LLM调用最多减少47倍。该系统并非单纯提升精度,而是通用编排层,可替代多个手工设计流程,仅凭自然语言生成合适执行策略。12个高文本密度查询的对照实验显示,BatchDAG得分为3.74/5,接近专家设计管道(3.25/5),显著优于ReAct代理(3.09/5,p<0.01),且溯源证据率高达77%,优于基线(46%-60%)。消融实验证明,结构化JSON中间态比自由文本摘要降低27%幻觉(配对t检验,p=0.107,n=12)。规划器在300次调用中达98.8%有效DAG率。在Brevian.ai生产环境中,系统可在60秒内处理超过5万次会议数据,每查询成本为$0.02-$0.24(按GPT-5.1定价)。
原文摘要 · Abstract (English)
Large language models (LLMs) excel at analyzing individual documents but break down on exhaustive, cross-entity analytical questions over enterprise-scale datasets due to context overflow, loss of per-entity attribution, and linear latency from sequential tool calls. We present BatchDAG, a system in which an LLM generates a typed directed acyclic graph (DAG) of operations -- SQL queries, semantic searches, in-memory transforms, parallel fan-outs, and single-shot analyses -- which a deterministic engine evaluates with topological-wave parallelism and structured JSON data flow. A key optimization, entity-aware batching, groups rows by logical entity before fan-out, reducing LLM calls by up to 47x. BatchDAG is not primarily an accuracy improvement over hand-optimized pipelines; rather, it is a general-purpose orchestration layer that replaces multiple hand-engineered workflows with a single system that generates the appropriate execution strategy from natural language. In controlled experiments on 12 transcript-heavy queries, BatchDAG (3.74/5) achieves quality comparable to an expert-designed pipeline (3.25/5) and significantly outperforms a ReAct agent (3.09/5, p<0.01), with superior provenance (77% transcript evidence rate vs. 46-60% for baselines). A controlled ablation shows structured JSON intermediates reduce hallucinations by 27% versus prose summaries (paired t-test, p=0.107, n=12). The planner achieves 98.8% valid-DAG rate across 300 planning calls. In production at Brevian.ai, BatchDAG processes queries over 50,000+ meetings in under 60 seconds, with measured per-query costs of $0.02-$0.24 at published GPT-5.1 pricing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。