用动态例子和生成检索提升大模型金融推理能力
Program of Thoughts for Financial Reasoning: Leveraging Dynamic In-Context Examples and Generative Retrieval
- 先生成检索相关事实，再动态选例进行程序化思考
- 在FinQA和ConvFinQA上准确率分别提升5.98%和4.05%
- 适合需要精准金融数据推理的场景或研究者
尽管大型语言模型（LLMs）能力持续进步，数值推理仍是难点。链式思维、树状思维和程序化思维提示虽能引导中间推理步骤，但少样本提示下的上下文学习仍使模型在金融数值推理数据集（如FinQA和ConvFinQA）上落后于顶尖水平。本文提出FINDER框架，分两步增强金融推理能力：首先使用生成式检索器从非结构化数据（文本与表格）中提取相关事实；随后采用上下文感知的程序化思维提示，并动态选择示例。FINDER在FinQA和ConvFinQA上均达新SOTA，执行准确率分别提升5.98%和4.05%。
原文摘要 · Abstract (English)
Despite continuous advancements in the capabilities of large language models (LLMs), numerical reasoning remains a challenging area. Techniques like chain-of-thought prompting, tree-of-thought prompting, and program-of-thought prompting guide LLMs through intermediate reasoning steps. Although in-context learning with few-shot prompting has improved performance, LLMs still lag behind state-of-the-art models on financial numerical reasoning datasets such as FinQA and ConvFinQA. In this work, we introduce FINDER, a novel two-step framework, to enhance LLMs' capabilities in financial numerical reasoning. The first step utilizes a generative retriever to extract relevant facts from unstructured data, including both text and tables. This is followed by context-aware Program of Thought prompting with dynamic selection of in-context examples. Our model FINDER achieves a new state-of-the-art performance on both the FinQA and ConvFinQA datasets, surpassing previous benchmarks with execution accuracy improvements of 5.98% and 4.05%, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。