无需训练的金融文档检索框架,提升信息提取效率与准确性。
PRISM: Prompt-Refined In-Context System Modelling for Financial Retrieval
- 通过优化提示词与上下文学习,实现高效文档排序。
- 在FinAgentBench上达NDCG@5 0.71818,位列前三且唯一无训练方案。
- 适合追求低部署成本与快速落地的金融智能应用开发者。
随着大语言模型的快速发展,金融信息检索已成为关键工业应用。从长篇财务文件中提取任务相关资讯,对运营与分析决策至关重要。我们提出PRISM,一种无需训练的框架,结合精炼系统提示、上下文学习(ICL)和轻量级多智能体协作,用于文档与片段排序任务。主要贡献在于系统性实证研究各组件的价值:提示工程带来稳定性能且开销极小;选择性使用ICL可增强复杂查询的推理能力;多智能体系统仅在大模型与精心设计架构下具潜力。在FinAgentBench、FiQA-2018和FinanceBench上的大量消融实验表明,简单配置常优于复杂多智能体流水线,为实践者提供实用指导。最佳配置在FinAgentBench上实现NDCG@5 0.71818,排名第三,且是该榜单前三位中唯一的无训练方法。我们还进行了延迟、令牌消耗与成本权衡的全面可行性分析,支持部署决策。源代码已发布于https://bit.ly/prism-ailens。
原文摘要 · Abstract (English)
With the rapid progress of large language models (LLMs), financial information retrieval has become a critical industrial application. Extracting task-relevant information from lengthy financial filings is essential for both operational and analytical decision-making. We present PRISM, a training-free framework that integrates refined system prompting, in-context learning (ICL), and lightweight multi-agent coordination for document and chunk ranking tasks. Our primary contribution is a systematic empirical study of when each component provides value: prompt engineering delivers consistent performance with minimal overhead, ICL enhances reasoning for complex queries when applied selectively, and multi-agent systems show potential primarily with larger models and careful architectural design. Extensive ablation studies across FinAgentBench, FiQA-2018, and FinanceBench reveal that simpler configurations often outperform complex multi-agent pipelines, providing practical guidance for practitioners. Our best configuration achieves an NDCG@5 of 0.71818 on FinAgentBench, ranking third while being the only training-free approach in the top three. We provide comprehensive feasibility analyses covering latency, token usage, and cost trade-offs to support deployment decisions. The source code is released at https://bit.ly/prism-ailens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。