专为金融推理设计的评分模型,提升大模型步骤与整体逻辑准确性。
Fin-PRM: A Domain-Specialized Process Reward Model for Financial Reasoning in Large Language Models
- 构建金融领域专用的分步与全程协同评分机制
- 在3000条金融推理轨迹上验证,显著优于通用模型
- 适合需要高可信推理的金融、量化分析场景
过程奖励模型(PRM)通过监督大语言模型的中间推理步骤来提升性能,但现有模型多基于通用数据训练,在金融推理这类结构化、符号化且依赖事实的任务中表现不佳。金融任务不仅要求最终答案正确,还需每一步都可验证并符合领域知识。本文提出Fin-PRM,一种面向金融推理的领域专用、轨迹感知的PRM,同时建模步骤级正确性与轨迹级连贯性,生成局部与全局推理质量的二值监督信号。为确保可靠监督,我们构建了一个包含3000条轨迹的高质量金融推理数据集,通过蒙特卡洛回溯、大模型评估和显式金融知识校验等多源奖励信号自动标注步骤与轨迹层级标签。Fin-PRM定义统一排序分数,融合步骤与轨迹奖励,支持多种场景应用。我们在三个场景中评估:(1)监督微调前的离线轨迹选择,(2)测试时扩展的Best-of-$N$推理,(3)强化学习中的过程感知奖励塑造。在CFLUE与FinQA等金融推理基准上的实验表明,Fin-PRM持续优于通用PRM与强基线。项目资源将公开于https://github.com/aliyun/qwen-dianjin。
原文摘要 · Abstract (English)
Process Reward Models (PRMs) supervise intermediate reasoning steps in large language models (LLMs), but existing PRMs are mainly trained on general-domain data and struggle with the structured, symbolic, and fact-sensitive nature of financial reasoning. Financial tasks require not only correct final answers but also verifiable intermediate steps grounded in domain knowledge. In this paper, we propose Fin-PRM, a domain-specialized, trajectory-aware PRM for financial reasoning that jointly models step-level correctness and trajectory-level coherence, producing binary supervision signals for both local and global reasoning quality. To support reliable supervision, we construct a high-quality financial reasoning dataset of 3K trajectories, where step- and trajectory-level labels are automatically derived from multi-source reward signals, including Monte Carlo rollouts, LLM-based evaluation, and explicit financial knowledge verification. Fin-PRM defines a unified ranking score that integrates step- and trajectory-level rewards, enabling consistent use across multiple settings. We evaluate Fin-PRM in three scenarios: (1) offline trajectory selection for supervised fine-tuning, (2) reward-guided Best-of-$N$ inference for test-time scaling, and (3) process-aware reward shaping for reinforcement learning. Experiments on financial reasoning benchmarks, including CFLUE and FinQA, show that Fin-PRM consistently outperforms general-purpose PRMs and strong baselines. Our project resources will be available at https://github.com/aliyun/qwen-dianjin.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。