对比两种LLM生成金融报告的方法,发现分解式更全面准确。
Template-Based Financial Report Generation in Agentic and Decomposed Information Retrieval
- 将报告模板拆解为多个查询,逐段由LLM回答
- 分解法在两个数据集上覆盖更广、细节更多
- 适合需要精准结构化报告的金融分析场景
从公司财报中生成结构化财务报告对理解财务表现至关重要,但现有摘要方法多生成宽泛的概述,缺乏财务报告所需的精确与细节。尽管大语言模型(LLMs)具有潜力,但生成符合预定义多段落模板的报告仍具挑战。本文研究了工业界流行的两种基于LLM的模板化报告生成方法:代理式信息检索(AgenticIR)与分解式信息检索(DecomposedIR)。AgenticIR通过提示完整模板驱动协作代理;DecomposedIR则采用提示链流程,将模板分解为子问题,每个部分作为查询由LLM基于财报回答。为量化评估,我们在两个场景下测试:一个使用无直接人工参考的金融数据集,另一个使用含专家撰写报告的气象数据集。实验结果表明,虽然AgenticIR在任务编排和生成简洁报告方面表现良好,但DecomposedIR在两个场景中均显著优于其,在内容覆盖范围和细节丰富度上更具优势,对代理框架在真实应用中的有效性提出反思。
原文摘要 · Abstract (English)
Tailoring structured financial reports from companies' earnings releases is crucial for understanding financial performance and has been widely adopted in real-world analytics. However, existing summarization methods often generate broad, high-level summaries, which may lack the precision and detail required for financial reports that typically focus on specific, structured sections. While Large Language Models (LLMs) hold promise, generating reports adhering to predefined multi-section templates remains challenging. This paper investigates two LLM-based approaches popular in industry for generating templated financial reports: an agentic information retrieval (IR) framework and a decomposed IR approach, namely AgenticIR and DecomposedIR. The AgenticIR utilizes collaborative agents prompted with the full template. In contrast, the DecomposedIR approach applies a prompt chaining workflow to break down the template and reframe each section as a query answered by the LLM using the earnings release. To quantitatively assess the generated reports, we evaluated both methods in two scenarios: one using a financial dataset without direct human references, and another with a weather-domain dataset featuring expert-written reports. Experimental results show that while AgenticIR may excel in orchestrating tasks and generating concise reports through agent collaboration, DecomposedIR statistically significantly outperforms AgenticIR approach in providing broader and more detailed coverage in both scenarios, offering reflection on the utilization of the agentic framework in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。