让大模型跨结构化与非结构化数据做多跳推理,还能追踪证据链。
Agentic DAG-Orchestrated Planner Framework for Multi-Modal, Multi-Hop Question Answering in Hybrid Data Lakes
- 将自然语言问题转为跨源多跳执行图(DAG),支持并行检索与依赖调度。
- 在基准测试中准确率提升14.8%,完整率提升10.7%,且输出可验证。
- 适合需要高可信度、跨数据湖问答的企业级应用,如金融审计、医疗知识库。
企业日益需要对融合结构化表格与非结构化文档的混合数据湖进行自然语言问答。现有部署方案(如RAG系统)通常依赖从各存储中暴力检索并事后合并,效率低、信息泄露严重,更关键的是缺乏对多跳推理的支持——即查询需分步拆解,在结构化与非结构化源间来回跳转。本文提出Agentic DAG-Orchestrated Transformer (A.DOT) Planner框架,将用户自然语言查询编译为跨越两类存储的有向无环图(DAG)执行计划。系统将查询分解为可并行的子查询,引入模式感知推理,并在执行前进行结构与语义验证。执行引擎严格遵循生成的DAG计划,协调异构源的并发检索,路由中间结果至依赖子查询,并按逻辑依赖合并最终结果。先进的缓存机制结合改写感知模板匹配,可识别等价查询并复用先前的DAG执行计划以快速重执行;DataOps系统则处理验证反馈或执行错误。该框架不仅提升准确率与延迟表现,还生成明确的证据链,支持内容溯源、数据血缘追踪,增强用户对输出的信任。在基准数据集上,A.DOT相比基线实现正确率绝对提升14.8%,完整率提升10.7%。
原文摘要 · Abstract (English)
Enterprises increasingly need natural language (NL) question answering over hybrid data lakes that combine structured tables and unstructured documents. Current deployed solutions, including RAG-based systems, typically rely on brute-force retrieval from each store and post-hoc merging. Such approaches are inefficient and leaky, and more critically, they lack explicit support for multi-hop reasoning, where a query is decomposed into successive steps (hops) that may traverse back and forth between structured and unstructured sources. We present Agentic DAG-Orchestrated Transformer (A.DOT) Planner, a framework for multi-modal, multi-hop question answering, that compiles user NL queries into directed acyclic graph (DAG) execution plans spanning both structured and unstructured stores. The system decomposes queries into parallelizable sub-queries, incorporates schema-aware reasoning, and applies both structural and semantic validation before execution. The execution engine adheres to the generated DAG plan to coordinate concurrent retrieval across heterogeneous sources, route intermediate outputs to dependent sub-queries, and merge final results in strict accordance with the plan's logical dependencies. Advanced caching mechanisms, incorporating paraphrase-aware template matching, enable the system to detect equivalent queries and reuse prior DAG execution plans for rapid re-execution, while the DataOps System addresses validation feedback or execution errors. The proposed framework not only improves accuracy and latency, but also produces explicit evidence trails, enabling verification of retrieved content, tracing of data lineage, and fostering user trust in the system's outputs. On benchmark dataset, A.DOT achieves 14.8% absolute gain in correctness and 10.7% in completeness over baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。