用角色分工的多智能体框架提升金融问答准确率
A Role-Aware Multi-Agent Framework for Financial Education Question Answering with LLMs
- 让生成、检索、评审三类智能体协作，按角色分工答题
- 相比零样本链式思考，准确率提升6.6%至8.3%
- 低成本实现接近专业模型效果，适合教育场景使用
问答在金融教育中至关重要，但现有大语言模型难以捕捉金融问题所需的复杂推理。金融领域需要多步定量分析、专业术语理解及现实情境把握。本文提出一种基于角色提示的多智能体框架，包含基础生成器、证据检索器和专家评审器，通过单轮迭代生成优化答案。在Study.com平台3,532道专家设计的金融教育题上评估，结合6本金融教材的RAG检索与专家提示策略。实验表明，基于批判的优化使准确率比零样本链式思考提升6.6%-8.3%，最高性能来自Gemini-2.0-Flash；且GPT-4o-mini表现可媲美微调后的FinGPT-mt_Llama3-8B_LoRA。结果展示了一种成本可控的金融QA增强方案，并为多智能体金融大模型研究提供洞见。
原文摘要 · Abstract (English)
Question answering (QA) plays a central role in financial education, yet existing large language model (LLM) approaches often fail to capture the nuanced and specialized reasoning required for financial problem-solving. The financial domain demands multistep quantitative reasoning, familiarity with domain-specific terminology, and comprehension of real-world scenarios. We present a multi-agent framework that leverages role-based prompting to enhance performance on domain-specific QA. Our framework comprises a Base Generator, an Evidence Retriever, and an Expert Reviewer agent that work in a single-pass iteration to produce a refined answer. We evaluated our framework on a set of 3,532 expert-designed finance education questions from Study.com, an online learning platform. We leverage retrieval-augmented generation (RAG) for contextual evidence from 6 finance textbooks and prompting strategies for a domain-expert reviewer. Our experiments indicate that critique-based refinement improves answer accuracy by 6.6-8.3% over zero-shot Chain-of-Thought baselines, with the highest performance from Gemini-2.0-Flash. Furthermore, our method enables GPT-4o-mini to achieve performance comparable to the finance-tuned FinGPT-mt_Llama3-8B_LoRA. Our results show a cost-effective approach to enhancing financial QA and offer insights for further research in multi-agent financial LLM systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。