用大模型规划阅读核安全文档，精准定位关键信息
LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents
- 构建基于大模型的智能代理，动态规划文档查阅路径
- 在200个问题上达81.5%准确率，显著优于其他方法38个百分点
- 可自动标记违规边，适合监管审查与人工审计场景
核安全文档审查需跨数万页进行多跳推理，判断依赖多章节证据整合。本文将任务建模为规划：基于LLM的智能体观察已有证据，选择下一步查阅的文档片段，并在证据充分时停止。该代理在无向量文档树上使用浏览、阅读和搜索工具，以动态知识图谱为状态。在NuScale最终安全分析报告（FSAR）的200题基准上，系统达到81.5%准确率，RAGAS忠实度为0.93。核心优势在于规划能力：相比不带状态决策的PageIndex（43.5%），性能提升38.0个百分点（p<0.001）。系统超越LightRAG（73.0%，p<0.05）、HippoRAG（70.5%，p<0.01）和GraphRAG（49.5%，p<0.001），与无需离线索引的RAPTOR（75.5%，p=0.11）相当。边缘推理增加2.8倍开销但未提高认知准确率，保留其作为可追溯性模块。共生成7,391条推断边，其中3条标注为'违反'（0.04%），提示范围越界（Q058）与部分符合（Q176），供人工审核。
原文摘要 · Abstract (English)
Reviewing nuclear regulatory documents requires multi-hop reasoning across tens of thousands of pages, where judgments depend on evidence assembled across multiple chapters. We frame this task as planning: an LLM-based agent observes the evidence collected so far, picks the next document fragment to inspect, and stops when the evidence is sufficient. The agent operates over a vectorless document tree using browse, read, and search tools, and maintains a dynamic knowledge graph (KG) as state. On a 200-question benchmark over NuScale Final Safety Analysis Report (FSAR) documents, the system reaches 81.5% accuracy with a RAGAS Faithfulness of 0.93. The dominant performance factor is planning: against PageIndex, which uses the same document tree without state-conditioned action selection, the gap is +38.0pp (43.5% to 81.5%, p<0.001). The system also outperforms LightRAG (73.0%, p<0.05), HippoRAG (70.5%, p<0.01), and GraphRAG (49.5%, p<0.001), and matches RAPTOR (75.5%, p=0.11) without offline indexing. Edge inference adds 2.8x cost without raising accuracy; we retain it as a traceability module. Of 7,391 inferred edges, 3 Violates edges (0.04%) flag scope boundaries (Q058) and partial conformance (Q176) as typed annotations that a human reviewer can audit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。