用动态证据树增强大模型，提升多文档分析推理能力
LLM Augmentations to support Analytical Reasoning over Multiple Documents
- 引入动态证据树记忆模块，追踪多条调查线索
- 实验证明原生大模型难以胜任复杂分析任务
- 适合情报分析、司法取证等需深度推理场景
基于大语言模型在多种任务中展现的能力，本文探究其在情报分析领域深化分析推理的应用。情报分析师常需处理海量档案，关联看似无关的实体，揭示对手计划与动机。我们研究大语言模型在此任务中的助益，并提出一种架构，通过称为动态证据树（Dynamic Evidence Trees, DETs）的记忆模块，增强大模型能力，以发展和追踪多个调查线索。在多个数据集上的大量实验表明，当前大语言模型本身仍不足以支持情报分析需求，并据此提出改进建议，以提升其在复杂推理应用中的表现。
原文摘要 · Abstract (English)
Building on their demonstrated ability to perform a variety of tasks, we investigate the application of large language models (LLMs) to enhance in-depth analytical reasoning within the context of intelligence analysis. Intelligence analysts typically work with massive dossiers to draw connections between seemingly unrelated entities, and uncover adversaries' plans and motives. We explore if and how LLMs can be helpful to analysts for this task and develop an architecture to augment the capabilities of an LLM with a memory module called dynamic evidence trees (DETs) to develop and track multiple investigation threads. Through extensive experiments on multiple datasets, we highlight how LLMs, as-is, are still inadequate to support intelligence analysts and offer recommendations to improve LLMs for such intricate reasoning applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。